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DIGITAL  MAP  REQUIREMENTS  STUDY  IN  SUPPORT  OF  ADVANCED 
COCKPIT  MOVING  MAP  DISPLAYS 


1.0  INTRODUCTION 

1.1  Objective 

The  objective  of  this  project  is  to  establish  the  map  data  requirements  for  a  next-generation 
digital  moving  map  system  that  will  be  designed  for  installation  in  the  F/A-18,  AV-8B,  AH-IW, 
UH-IN,  V-22,  and  potentially  other  aircraft.  A  primary  Naval  Air  Systems  Command  (NAVAIR) 
goal  in  specifying  the  new  system  is  to  enhance  situational  awareness  (SA)  and  aircrew  mission 
effectiveness  without  further  burdening  pilot  task  workload.  To  ensure  that  the  end-users’  explicit 
map  needs  are  taken  into  consideration,  investigators  elicited  one-on-one  aircrew  evaluations  of  a 
wide  variety  of  map  data  types  (both  topographic  and  tactical)  and  map  display  parameters,  including 
feature  size,  orientation,  color,  symbology,  etc.,  to  help  define  an  optimum  map  design  for  cockpit 
displays. 


1.2  Background 

In  support  of  this  objective,  the  Tactical  Aircraft  Moving  Map  Capability  (TAMMAC)  Integrated 
Program  Team  (IPT)  at  NAVAIR  (PMA  209)  funded  investigators  from  the  Naval  Research  Laboratory, 
Stennis  Space  Center,  MS  (NRLSSC),  to  identify  and  demonstrate  which  digital  map  products 
would  be  best  suited  to  support  end-user  requirements  for  advanced  cockpit  map  displays.  The 
results  of  this  work  may  benefit  mission  planning  displays  as  well,  although  mission  planners  are 
not  explicitly  targeted  in  this  study. 

NRLSSC  designed  16  task-structured  demonstrations  of  various  digital  moving  map  scenarios, 
using  standard  National  Imagery  and  Mapping  Agency  (NIMA)  products  (including  both  existing 
and  prototype  databases),  and  presented  the  displays  to  experienced  aircrew  from  diverse  aircraft 
platforms.  We  asked  the  pilots  to  evaluate  each  moving  map  display  in  terms  of  its  potential 
usefulness  for  their  specific  applications. 

We  attempted  to  simulate  realistic  mission  scenarios,  but  due  to  time  and  funding  constraints, 
we  were  unable  to  incorporate  the  map  displays  into  a  flight  simulator.  Instead,  we  developed  and 
presented  the  demonstrations  on  a  Silicon  Graphics,  Inc.  (SGI)  workstation.  We  conducted  the 
demonstrations  and  aircrew  surveys  at  the  Naval  Air  Warfare  Center,  Aircraft  Division,  Patuxent 
River,  MD  (NAWC-ADP),  in  conjunction  with  their  Human  Factors  group. 


1.3  Order  of  Report 

This  report  documents  the  moving  map  demonstrations  that  were  presented  to  pilots  and  aircrew 
at  Patuxent  River  in  August  1995,  and  presents  the  results  of  the  coincident  aircrew  survey. 
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Following  this  Introduction  section,  an  Approach  and  Methods  section  describes  the  in-depth 
questionnaire,  interview  sessions,  and  an  overview  of  each  demonstration,  including  hardware, 
software,  and  map  data  sources.  The  Results  and  Recommendations  section  of  the  report  discusses 
each  demonstration  in  detail,  including  a  short  description,  summary  of  pertinent  NAVAIR  program 
requirements,  survey  results,  and  specific  recommendations.  The  report  closes  with  a  Summary, 
References,  and  Appendices,  which  provide  additional  information  on  specific  aircraft  map 
requirements  and  the  pilot  surveys. 


2.0  APPROACH  AND  METHODS 

2.1  Questionnaire 

The  aircrew  questionnaire  consisted  of  a  pilot  introduction  page,  followed  by  one  survey  for 
each  of  the  16  demonstrations.  All  responses  were  entered  into  a  database  (Microsoft  Access  2.00) 
for  compilation  and  data  analysis.  The  entire  questionnaire  is  provided  in  App.  A. 

The  pilot  introduction  section  included  information  such  as  military  service,  primary  and  secondary 
aircraft,  flight  hours,  night  flight  experience,  combat  experience,  and  experience  with  existing 
cockpit  digital  moving  maps.  This  information  was  used  to  categorize  the  results  and  to  consider 
whether  a  particular  type  of  map  data  or  display  would  be  more  useful  for  one  type  of  aircraft,  or 
whether  a  pilot’s  responses  might  be  related  to  his  or  her  flight  experience  or  familiarity  with 
digital  moving  maps. 

Each  map  display  survey  started  with  a  brief  description  of  the  demo,  followed  by  a  series  of 
questions.  The  surveys  were  designed  to  be  as  quantitative  as  possible  to  facilitate  data  entry  and 
analysis.  Most  questions  required  answers  in  one  of  the  following  forms: 

•  5-point  ranking  of  map  data  or  display  items  (Table  1); 

•  Rating  of  items  as  either  satisfactory  or  unsatisfactory; 

•  Multiple  choice  (e.g.,  “Which  option(s)  would  you  prefer:  A,  B,  and/or  C  ?”); 

•  Questions  requiring  short,  one-  to  two-sentence  answers. 

2.2  Interview  Sessions 

NRLSSC  interviewers  conducted  the  demonstration  for  each  pilot  separately  to  encourage 
individual  responses.  Each  pilot  viewed  a  6-min  introductory  video,  followed  by  the  demonstrations 

and  questionnaire,  then  a  brief  wrap-up.  The  entire  session 
typically  lasted  Vh  h  per  pilot. 

The  introductory  video  gave  each  pilot  an  overview  of 
this  project,  the  chief  objectives,  and  some  examples  of  the 
demonstrations  they  would  be  viewing  and  the  questions  they 
would  be  asked.  The  participants  also  filled  out  the  pilot 
introduction  sheet  at  this  time. 

Each  of  the  16  demonstrations  was  between  1-5  min  long. 
For  each  demonstration,  the  interviewer  asked  the  pilot  to 
read  the  brief  description  and  all  questions  on  that  demo’s 


Table  1  —  Demo  Rating  Scale 


RATING 

DESCRIPTION 

1 

Of  No  Use 

2 

Not  Very  Useful 

3 

Of  Use 

4 

Of  Considerable  Use 

5 

Extremely  Useful 
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survey  sheet  before  viewing  the  demo.  The  pilot  could  view  the  demo  more  than  once,  if  desired. 
Each  pilot  could  take  as  much  time  as  necessary  to  answer  all  the  questions  about  one  demo  before 
viewing  the  next.  Each  session  was  tape-recorded  (with  the  participant’s  knowledge)  to  capture  any 
comments  that  the  pilot  may  not  have  written  on  the  questionnaire. 

Finally,  each  pilot  was  given  about  10  min  to  wrap-up  and  make  any  additional  comments 
about  the  demos  or  digital  moving  map  displays  in  general.  In  an  attempt  to  focus  the  pilot’s  final 
comments,  the  interviewer  asked  “If  you  were  to  go  into  combat  tomorrow,  is  there  one  item  from 
these  demonstrations  that  you  would  want  to  take  with  you?”  It  was  anticipated  that  their  responses 
might  shed  some  light  on  bottom-line  priorities. 


2.3  Demonstrations 

The  demonstrations  were  developed  as  computer-generated  movie  loops  using  Arcinfo  Geographic 
Information  System  software  and  SGI  Moviemaker  software  on  SGI  Crimson  and  Indigo  workstations. 
Realistic  ground  speeds,  aircraft  turn  rates,  display  refresh  rates,  and  other  parameters  were  simulated 
via  Moviemaker  by  carefully  controlling  the  window  of  map  data  displayed  in  each  frame  (including 
exact  geographic  area,  image  orientation,  zoom  factor,  etc.)  and  the  number  of  frames  displayed  per 
second.  Arcinfo  handled  map  projection  and  scale  compatibility  (between  overlaid  data  sets).  The 
simulated  map  display  window  was  the  same  size  as  the  current  F/A-18  and  AV-8B  digital  map 
displays,  4.5"  x  4.5"  (11.4  cm  x  11.4  cm). 

Each  of  the  16  demonstrations  addressed  one  or  more  specific  map  data  or  display  issues  of 
particular  interest  to  the  TAMMAC  team  at  NAVAIR.  NAWC-ADP  and  NRLSSC  developed  a 
matrix  of  46  map  issues  from  the  TAMMAC  Requirements  Database,  which  NAWC-ADP  compiled 
from  the  sources  listed  in  Table  2.  Of  the  46  original  map  issues,  24  were  considered  to  be  within 
the  scope  of  This  project  and  were  evaluated  during  this  study.  This  subset  of  24  map  issues  is 
presented  in  App.  B. 

Based  on  the  TAMMAC  requirements,  six  principal  map  data  types  were  selected  for  evaluation; 
scanned  chart  data,  satellite  imagery,  terrain  elevation  data,  data  frames  (such  as  reconnaissance 
photographs),  vector  map  data,  and  mission  planning  symbols.  Table  3  provides  descriptions  and 
source  information  for  each  of  these  data  types. 


Table  2  —  Sources  for  TAMMAC  Map  Data  Requirements  by  Aircraft  Platform 


AIRCRAFT 

MAP  DATA  REQUIREMENTS  SOURCES 

AV-8B 

F/A-18  C/D 

F/A-18  E/F 

UH-IN 

V-22 

PS  75-870134  Rev.  A  (Procurement  Spec  for  Digital  Map  Set),  29  Oct  85 

PS  75-870134  Rev.  A  (Procurement  Spec  for  Digital  Map  Set),  29  Oct  85 

PS  74-870300  (Procurement  Spec  for  Multi-Purpose  Color  Display),  27  Jan  93 

System  Spec  for  UH-IN  Multi-Function  Display,  NAWCADWAR,  14  Apr  94 

V-22  Draft  DMS  Spec  #901-947-855,  14  Mar  94;  Digital  Map  System  CIDS,  Boeing 
Memo  8-7161-DM-276,  5  Apr  94;  Supplier  Statement  of  Work  for  DMS,  Boeing 

Report  #D901-99549-l,  1994;  V-22  DMS,  Vol.  II  -  Eng.  Doc  #HP94-0501,  Elbit,  Fort 
Worth,  TX,  17  Jun  94 
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Table  3  —  Data  Selected  for  Cockpit  Moving  Map  Demonstrations 


DATA  TYPE 


Scanned  Chart 
Data 


DATABASE  NAME  AND  DESCRIPTION 


ARC  Digitized  Raster  Graphics  (ADRG),  subsampled  to 
169  pixels/inch  to  emulate  Compressed  ADRG  (CADRG), 
including  the  following  scanned  aeronautical  charts: 


Acronym 

Scale* 

Ranged 

Full  name 

GNC 

1:5M 

160  nmi 

Global  Navigation  Charts 

JNC 

1:2M 

80  nmi 

Jet  Navigation  Charts 

ONC 

1:1M 

40  nmi 

Operational  Navigation  Charts 

TPC 

1:500k 

20  nmi 

Tactical  Pilotage  Charts 

JOG 

1:250k 

10  nmi 

Joint  Operational  Graphics 

TLM-100 

1:100k 

4  nmi 

Topographic  Line  Map-100 

TLM-50 

1:50k 

2  nmi 

Topographic  Line  Map-50 

Satellite  Imagery 


*For  scales:  M  =  million,  k  =  thousand. 

^Range  based  on  current  cockpit  moving  map  displays 
(McDonnell  Douglas  Model  #ASQ-196):  top  to  bottom  of 
screen. 

Controlled  Image  Base  (CIB):  10  m/pixel  panchromatic 
Satellite  Pour  I’Observation  de  la  Terre  (SPOT)  imagery 
enhanced  via  contrast-stretch  algorithm  (in  Arcinfo). 
Equivalent  scale:  1:50k  (2  nmi  range). 


Vector  Map  Data  Digital  Chart  of  the  World  (DCW):  vectorized  version  of  the 
1:1M  scale  ONC  series  (40  nmi  range,  pre-zoom). 

Terrain  Data.  Digital  Terrain  Elevation  Data  (DTED)  Level  1:  gridded 
elevation  database  (one  grid  point  every  3  arc-seconds  of 
latitude).  Equivalent  scale:  1:250k  (10  nmi  range). 

Data  Frames  Any  static  image,  such  as  a  reconnaissance  photograph, 

emergency  checklist,  etc.  Samples  used  in  this  study  were 
reconnaissance  photos  at  varying  scales. 

Mission  Planning  Bitmapped  (raster)  versions  of  cartographic  symbols 


Symbols 


representing  threats,  targets,  routes,  waypoints,  etc.  Included 
symbols  from  AV-8B  and  V-22  mission  planning  sets. 


TAMPS  =  Tactical  Aircraft  Mission  Planning  System 
MOMS  =  Map,  Operator,  and  Maintenance  Stations 


SOURCE(S) 

NIMA 


NIMA 


NIMA 


NIMA 


NIMA 


TAMPS,  MOMS 


China  Lake,  CA,  was  chosen  as  the  geographic  area  of  interest  for  the  simulated  moving  map 
displays  primarily  because  all  the  map  databases  to  be  demonstrated  included  coverage  of  that  area. 
The  CIB  database,  in  particular,  had  very  limited  geographic  coverage,  but  it  did  include  China  Lake. 

Throughout  the  demonstrations,  ground  speed  was  simulated  at  several  different  velocities, 
ranging  from  0  to  480  kt,  to  consider  how  the  moving  map  might  behave  in  various  aircraft 
(e.g.,  hovering  or  slower  moving  helicopters,  as  well  as  faster  tactical  aircraft).  Turns  were 
simulated  at  7  degrees  per  second  (deg/s)  unless  otherwise  stated. 


TAMMAC  Digital  Map  Requirements  Study 


5 


The  16  demonstrations  were  grouped  into  six  general  categories,  based  on  the  map  data  and 
display  issues  to  be  addressed;  timing,  map  positioning,  zooming,  terrain  elevation  data,  overlay 
data,  and  future  moving  map  displays.  Table  4  lists  each  demo  by  category. 


3.0  PILOT  PROFILE 

This  section  presents  the  results  of  the  pilot  information  surveys,  including  aircraft  platforms 
represented,  military  services  represented,  and  pilot  experience  (e.g.,  number  of  flight  hours,  combat 
experience,  and  digital  moving  map  experience). 


3.1  Aircraft  and  Services  Represented 

NRLSSC  and  NAWC-ADP  interviewed  a  total  of  30  pilots,  representing  14  different  aircraft 
platforms  (Table  5)  from  the  Navy,  Marine  Corps,  and  Air  Force  (Fig.  1).  Although  the  original 
intent  was  to  survey  only  tactical  pilots,  several  nontactical  groups  were  also  represented  (e.g., 
helicopter  pilots,  antisubmarine  warfare  (ASW)  pilots,  and  aircrew).  In  hopes  that  this  diversity 
would  shed  insight  on  the  potential  differences  in  map  data  requirements  as  a  function  of  aircraft 
type  and  mission,  many  of  the  survey  results  were  categorized  by  general  aircraft  category  {Tactical, 
Helicopter,  and  ASW,  as  listed  in  Table  5). 

Participants  were  asked  to  list  their  primary  and  secondary  aircraft;  however,  this  report  bases 
its  results  and  recommendations  on  primary  aircraft  only.  A  larger  sample  size  would  be  required 
before  responses  could  be  successfully  linked  to  more  than  one  aircraft  per  participant  with  any 
consistent  results. 


Table  4  —  List  of  All  Demonstrations  by  Category 


TIMING 

(1)  Time  to  Switch  Map  Modes 

(2)  Time  to  Switch  Scales 

(3)  Time  to  Reposition  to  Lat/Lon 

(4)  Data  Update/Refresh  Rates 

TERRAIN  ELEVATION  DATA 

(10)  Contours  vs.  Shading 

(11)  Contour  Intervals 

(12)  Sun-Angle  Shading  (2-D  and  3-D) 

MAP  POSITIONING 

(5)  Centered  vs.  Decentered  in  North-Up  and 
Track-Up 

OVERLAY  DATA 

(13)  Height  Above  Terrain  (HAT)  Over  Chart 
or  Imagery 

(14)  Clear  Line  of  Sight  (CLOS)  Over 

Imagery 

(15)  Threat  Intervisibility 

ZOOMING 

(6)  Zoom  vs.  Chart  Series  Switches 

(7)  Number  of  Steps  to  Zoom 

(8)  Zoom-Out  Capability 

(9)  Range  vs.  Legibility  Issues 

VECTOR  MAPS 

(16)  Vector  Moving  Map  Display 
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Fig.  1  —  Military  services 
represented  in  survey 


Table  5  —  Aircraft  Types  Represented  in  Survey 


TACTICAL:  15 

HELICOPTER:  9 

ASW:  6 

A/C 

#PILOTS 

A/C 

#PILOTS 

A/C 

#PILOTS 

F/A-18 

6 

AH-IW 

2 

P-3 

3 

A-6 

2 

CH-46 

2 

S-3 

2 

AV-8B 

2 

CH-53E 

2 

V-22 

1 

EA-7 

2 

UH-IN 

2 

F-14 

2 

H-60 

1 

A-10* 

1 

The  A-10  pilot  was  interviewed  during  a  preliminary  survey. 
He  did  not  respond  to  all  questions,  resulting  in  a  total  of  14 
tactical  pilots  (29  total  pilots)  for  some  demos. 


3.2  Pilot  Experience 

The  pilot  introduction  survey  gauged  pilot  experience  in  several  ways:  number  of  flight  hours, 
percentage  of  hours  flown  at  night,  combat  flight  experience  {none;  limited',  or  experienced),  flight 
instructor  experience  (yes  or  no),  and  digital  moving  map  experience  {no  experience;  familiar  with 
concept;  limited  experience;  occasional  use;  or  current  and  experienced  user). 


3.2.1  Flight  Hours 

The  majority  of  surveyed  pilots  had  logged  between  1000-3000  total  flight  hours,  including 
between  1000-2000  h  in  their  primary  aircraft  (Fig.  2).  The  least  experienced  participants  (in  terms 
of  total  flight  hours)  were  two  EA-7  aircrew  members,  who  each  had  1100  total  flight  hours  (including 
600  h  each  in  the  EA-7).  The  most  experienced  participant  was  a  P-3  pilot  with  6000  total  hours 
(including  1000  in  the  P-3).  The  average  number  of  total  flight  hours  cited  was  2400;  the  average 
number  of  flight  hours  in  pilots’  primary  aircraft  was  1490. 
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Fig.  2  —  Flight  hours  for  surveyed  pilots:  (a)  total  number  of  flight  hours  and  (b)  number  of 

flight  hours  in  primary  aircraft 
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3.2.2  Night  Flying  Experience 


The  survey  recorded  nighttime  flight  hours  as  well.  The  percentage  of  flight  hours  that  were 
at  night  ranged  from  5-50%  (or  55-3000  h)  for  this  population,  with  an  average  of  just  over  30% 
(860  h).  The  number  of  night  flight  hours  was  determined  by  applying  each  pilot’s  night  flight 
percentage  to  his  or  her  total  flight  hours. 


3.2.3  Combat  Experience 

^  The  survey  asked  pilots  to  describe  their  own  combat  flight  experience  by  selecting  one  of  the 

following  options:  (1)  No  combat  experience;  (2)  limited  combat  experience;  or  (3)  experienced 
combat  pilot.  Table  6  shows  that  half  of  the  surveyed  pilots  had  no  combat  experience  and  one-third 
had  limited  experience.  Only  three  respondents  rated  themselves  experienced  combat  pilots. 

#  3.2.4  Flight  Instructors 

The  survey  recorded  whether  each  pilot  was  a  certified  flight  instructor  as  another  indication 
of  overall  flight  experience.  Of  the  30  pilots  interviewed,  17  were  instructors  (57%),  suggesting  a 
relatively  mature  and  experienced  pilot  population. 

^  3.2.5  Digital  Moving  Map  Experience 

Finally,  the  survey  asked  participants  to  assess  their  experience  with  cockpit  moving  maps  by 
selecting  one  of  the  following  options:  (1)  No  experience;  (2)  familiar  with  concept;  (3)  limited 
experience  (e.g.,  in  a  simulator);  (4)  occasional  use  in  cockpit;  or  (5)  current  use  and  very  experienced. 

^  As  shown  in  Table  7,  77%  of  the  pilots  in  this  sample  had  some  experience  with  cockpit  moving  maps, 

54%  had  flown  (frequently  or  occasionally)  with  a  cockpit  moving  map,  and  23%  had  limited  experience. 
Another  20%  X)f  the  pilots  had  no  first-hand  experience  with  moving  maps,  but  were  familiar  with 
the  concept.  Only  one  pilot  claimed  to  have  no  experience  with  this  technology.  These  responses 
suggest  a  fairly  sophisticated  pilot  population  that  is  familiar  with  digital  cockpit  moving  maps. 


4.0  RESULTS  AND  RECOMMENDATIONS 

This  section  presents  the  results  of  the  demonstration  surveys,  including  a  brief  description  of 
each  demonstration  and  any  known  preliminary  aircraft  program  specifications  related  to  the  demo 


m. 


Table  6  —  Combat  Experience  of 
Participants 


DESCRIPTION 

#  PILOTS 

No  Combat  Experience 

15  (50%) 

Limited  Combat  Experience 

10  (33%) 

Experienced  Combat  Pilot 

3  (10%) 

No  Answer 

2  (7%) 

Table  7  —  Digital  Moving  Map  Experience  of 
Participants 


DESCRIPTION 

#  PILOTS 

Current  Use  and  Very  Experienced 

5  (17%) 

Occasional  Use  in  Cockpit 

11  (37%) 

Limited  Experience  (e.g.,  in  a  simulator) 

7  (23%) 

Familiar  With  Concept 

6  (20%) 

No  Digital  Moving  Map  Experience 

1  (3%) 
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(also  listed  in  App.  B).  The  discussion  of  demonstration  surveys  is  organized  by  demo  category 
(Table  4)  and  provides  category-specific  survey  results  and  recommendations  following  the  demo 
descriptions. 


4.1  Timing  Requirements 

4.1.1  Description 

The  first  three  demos  in  this  category  demonstrated  varying  times  to  switch  the  map  display 
from  one  view  to  another.  As  a  switch  started  in  each  demo,  a  programmed  sound  bite  in  the 
computer  said  “switching.”  To  illustrate  the  amount  of  time  taken  to  perform  the  switch,  the  display 
turned  black  the  instant  a  pilot  made  the  “request,”  and  it  returned  to  its  original  color  the  instant 
the  display  was  changed.  The  request  in  each  case  was  simulated;  none  of  the  demos  was  interac¬ 
tive.  Participants  were  not  told  how  much  time  each  switch  took;  instead,  they  were  told  to  evaluate 
each  switch  time  as  either  satisfactory  or  unsatisfactory. 

The  last  demo  in  this  category  evaluated  data  update  rates,  or  the  speed  at  which  the  map 
computer  displays  each  new  frame  of  information.  The  display  update  rate  determines  how  well  the 
moving  map  display  can  “keep  up”  with  the  speed  of  the  aircraft.  In  all  four  timing  demos,  chart 
data  refers  to  subsampled  ADRG  JOG  data  (see  Table  3)  unless  otherwise  indicated. 

Demo  #i;  Time  to  Switch  Map  Modes:  Demonstrated  three  different  delay  times  to  switch 
between  map  modes  (or  data  types):  1  s  (switch  time  A),  2  s  (B),  and  3  s  (C).  Preliminary  speci¬ 
fications  for  the  V-22  program  called  for  a  switch  time  of  1  s  or  less  for  this  function;  the  preliminary 
UH-IN  specification  was  3  s  or  less  (App.  B,  items  8,  11,  and  39).  For  each  switch  time,  the  map 
display  changed  from  a  JOG  chart  to  satellite  imagery  to  terrain  data  to  a  data  frame  (Fig.  3a-d). 
Each  map  mode  was  displayed  for  5  s  before  switching  to  the  next  mode. 

The  demo  displayed  the  first  three  modes  as  north-up  moving  maps;  the  data  frame  was  a  static 
image.  The  JOG  chart  and  terrain  data  were  displayed  at  a  scale  of  1:250k  (which  equates  to  10  nmi 
from  screen  top  to  bottom),  and  the  satellite  imagery  was  displayed  at  a  scale  of  1:50k  (2  nmi  range). 
The  data  frame  was  a  reconnaissance  photograph  (approximately  0.5  nmi  range). 

Demo  #2;  Time  to  Switch  Scales:  Demonstrated  three  different  delay  times  to  switch  between 
chart  scales:  0.5  s  (switch  time  A),  1  s  (B),  and  2  s  (C).  Preliminary  UH-IN  specifications  called 
for  a  switch  time  of  1  s  or  less  for  this  function  (App.  B,  item  6).  For  each  switch  time,  the  map 
display  changed  from  JOG  (10  nmi  range)  to  TPC  (20  nmi  range)  and  back  to  JOG  (Fig.  4).  Each 
chart  series/scale  was  displayed  for  5  s  before  switching  to  the  next  series.  All  displays  were 
north-up  moving  maps. 

Demo  #3:  Time  to  Switch  from  Moving  Map  to  Command  LatILon  Reposition:  Demonstrated 
three  different  times  to  reposition  a  moving  map  to  a  specified  point:  0.5  s  (switch  time  A),  1  s  (B), 
and  2  s  (C).  The  UH-IN  program  specified  a  switch  time  of  1  s  or  less  for  this  function  (App.  B, 
item  32).  For  each  switch  time,  the  display  switched  from  a  moving  chart  centered  on  aircraft 
position  to  a  static  chart  centered  on  a  “pilot-specified”  latitude  and  longitude  point  (outside  the 
original  display  range),  and  back  to  the  aircraft-centered  moving  chart  (Fig.  5).  Each  view  was 
shown  for  5  s  before  switching  to  the  next  view. 

Demo  #4:  Data  Updatel Refresh  Rates:  Compared  two  different  data  update  rates  (15  and 
20  Hz)  for  varying  aircraft  speeds.  The  V-22  program  specified  a  data  update  rate  of  15  Hz; 


# 


Fig.  3  —  Demo  1  of  four  map  modes,  centered,  north-up  display:  (a)  CADRG 
9  data,  JOG  chart,  10  nmi  range  and  (b)  satellite  imagery,  CIB/SPOT,  2  nmi  range 
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Fig.  3  —  (cont.)  (c)  terrain  elevation  data,  DTED,  10  nmi  range  and  (d) 
dataframe,  reconnaissance  photo,  0.5  nmi  range 
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Fig.  4  —  Demo  2:  switching  between  two  chart  scales,  centered,  north-up  displays:  (a)  JOG  (1:250k  scale)  and 

(b)  TPC  (1:500k  scale) 


Fig.  5  —  Demo  3:  switching  from  moving  map  to  repositioned  chart,  centered,  north-up  displays  (both  JOG  1:250k 

scale):  (a)  moving  map  and  (b)  repositioned  chart 


preliminary  TAMMAC  specifications  called  for  20  Hz  (App.  B,  item  1).  A  rate  of  15  Hz  was 
simulated  by  displaying  15  frames/s  (a  new  frame  was  displayed  every  66  ms),  and  20  Hz  was  simulated 
by  displaying  20  frames/s  (a  new  frame  was  displayed  every  50  ms).  For  each  update  rate, 
ground  speed  was  simulated  at  0,  90,  200,  and  300  kt  with  the  aircraft  in  a  7  deg/s  turn. 

4.1.2  Results 

Tables  8  and  9  present  the  results  of  the  timing  studies.  In  the  first  demo  (which  switched 
among  four  different  “map  modes”  or  data  types),  switch  times  of  1  s,  2  s,  and  3  s  were  presented. 
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Table  8  —  Acceptance  Values  for  Switch  Times 
(Demos  1,  2,  and  3) 
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ACCEPTANCE  OF  DELAY 

(%  of  Surveyed  Pilots  that 

Considered  the  Delay  Acceptable) 

DEMO  #/DESCRIPTION 

0.5  s 

1  s 

2  s 

3  s 

Switch/Map  Modes 

- 

100% 

40% 

3% 

Tactical 

— 

15/15 

5/15 

0/15 

Helicopter 

- 

9/9 

5/9 

1/9 

ASW 

- 

6/6 

2/6 

0/6 

Switch  Chart  Scales 

100% 

77% 

13% 

- 

Tactical 

15/15 

9/15 

3/15 

— 

Helicopter 

9/9 

8/9 

1/9 

- 

ASW 

6/6 

6/6 

0/6 

- 

Command  Reposition 

100% 

83% 

20% 

- 

Tactical 

15/15 

11  /  15 

4/15 

— 

Helicopter 

9/9 

8/9 

1/9 

- 

ASW 

6/6 

6/6 

1/6 

- 

Table  9  —  Acceptance  Values  for  Data  Update  Rates  (Demo  4) 


ACCEPTANCE  OF  DATA  UPDATE  RATES 

(%  of  Surveyed  Pilots  that  Considered  the  Rate  Acceptable) 

DEMO  #4  DESCRIPTIONS 

15  Hz 

20  Hz 

NEITHER 

Data  Update  Rates  @  0  kt 

97% 

83% 

0% 

Tactical 

13/14 

11/14 

0/14 

Helicopter 

9/9 

7/9 

0/9 

ASW 

6/6 

6/6 

0/6 

Data  Update  Rates  @  90  kt 

97% 

76% 

0% 

Tactical 

13/14 

10/14 

0/14 

Helicopter 

9/9 

6/9 

0/9 

ASW 

6/6 

6/6 

0/6 

Data  Update  Rates  @  200  kt 

97% 

69% 

0% 

Tactical 

13/14 

10/14 

0/14 

Helicopter 

9/9 

4/9 

0/9 

ASW 

6/6 

6/6 

0/6 

Data  Update  Rates  @  300  kt 

93% 

72% 

3% 

Tactical 

13/14 

10/14 

0/14 

Helicopter 

8/9 

5/9 

1/9 

ASW 

6/6 

6/6 

0/6 
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All  surveyed  pilots  accepted  the  1-s  delay  as  satisfactory;  less  than  half  of  the  pilots  accepted  a 
2-s  delay,  and  only  one  pilot  accepted  a  3-s  delay. 

In  both  the  second  demo  (switching  between  chart  scales)  and  the  third  demo  (switching  from 
a  moving  map  to  a  command-repositioned  chart),  switch  times  of  0.5  s,  1  s,  and  2  s  were  presented. 
Again,  all  surveyed  pilots  accepted  the  fastest  time  switch,  which  was  0.5  s  in  these  cases. 
Approximately  80%  of  the  pilots  accepted  a  1-s  delay  for  scale  switches  and  for  command 
repositions,  as  opposed  to  the  100%  acceptance  of  a  1-s  delay  to  switch  map  modes.  Only  13%  and 
20%  of  the  pilots  accepted  a  2-s  delay  for  scale  switches  and  command  repositions,  respectively, 
as  opposed  to  a  40%  acceptance  for  a  2-s  delay  to  switch  map  modes. 

There  are  at  least  two  possible  explanations  for  the  disparity  of  acceptance  between  the  first 
demo  and  the  second/third  demos,  with  regard  to  the  1-  and  2-s  switches.  First,  each  demo  pre¬ 
sented  these  time  delays  to  the  pilots  in  order  of  fastest  to  slowest.  Although  the  pilots  were  not 
informed  of  this  pattern,  they  may  have  learned  to  anticipate  progressively  slower  switches  and 
chose  the  fastest  (the  first  option  in  each  demo)  as  the  most  acceptable.  Similarly,  the  second  option 
in  each  demo  tended  to  be  less  acceptable  than  the  first,  since  it  was  obviously  slower,  and  the  last 
option  tended  to  be  the  least  acceptable,  since  it  was  perceived  to  be  the  worst  case.  This  bias  could 
be  minimized  in  future  surveys  by  randomizing  the  presentation  of  switch  times. 

A  second  possible  explanation  for  the  disparity  of  acceptance  between  the  demos  is  that  the 
participants  could  tolerate  a  longer  delay  in  switching  between  different  map  modes  as  compared 
to  switching  between  chart  scales  or  command  repositions.  This  explanation  is  supported  by  the 
significant  difference  in  acceptance  of  a  1-s  delay  between  tactical  pilots  and  all  other  pilots.  All 
pilots  considered  a  1-s  delay  to  be  acceptable  when  switching  map  modes,  and  all  but  one  of  the 
nontactical  pilots  rated  this  delay  acceptable  for  switching  scales  and  switching  to  a  command 
repositioned  area.  However,  only  two-thirds  of  the  tactical  pilots  considered  a  1-s  delay  acceptable 
for  switching  scales  and  repositioning. 

More  tactical  and  helicopter  pilots  rated  the  15-Hz  data  update  rate  acceptable  than  the  20-Hz 
display  at  the  four  demonstrated  ground  speeds  (a  chi-square  test  showed  their  preference  for 
15  Hz  to  be  significant  at  a  =  0.01).  ASW  pilots  found  no  difference  between  the  two  data  update 
rates.  These  results  were  unexpected,  since  it  was  theorized  that  the  faster  display  (20  Hz)  would 
be  better.  Once  again,  the  pilots  may  have  been  conditioned  to  expect  that  the  first  segments  of  a 
demo  were  the  best  (fastest)  and  that  later  segments  were  less  acceptable.  Nevertheless,  even  pilots 
who  viewed  both  options  (15  Hz  and  20  Hz)  several  times  could  not  identify  a  significant  difference 
between  the  speeds,  and  more  pilots  rated  15  Hz  as  acceptable  than  20  Hz. 

Pilots’  written  (and  taped)  comments  provide  additional  information  about  data  update  rate 
requirements.  Pilots  stated  that  “Time  A  [15  Hz]  was  smoother,”  and  “Time  B  [20  Hz]  jittered 
more”  or  was  “rachety.”  At  least  one  pilot  (V-22)  stated  that  the  preferred  update  rate  (15  Hz)  must 
be  maintained  while  displaying  all  mission  planning/waypoint  symbols  on  the  map.  He  stated  that 
this  is  a  problem  with  current  map  displays;  the  map  has  trouble  “keeping  up”  with  the  aircraft 
position  after  all  the  necessary  symbols  have  been  overlaid.  Two  other  pilots  (UH-IN  and  AH-IW) 
cautioned  that  neither  of  the  simulated  data  update  rates  (15  Hz  nor  20  Hz)  may  be  acceptable 
during  a  typical  helicopter  turn  (20-40  deg/s  as  opposed  to  the  demonstration’s  turn  rate  of 
7  deg/s),  and  that  update  rates  should  be  demonstrated  at  faster  turns  before  they  could  accurately 
evaluate  them. 
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4.1.3  Summary  and  Recommendations 

Times  to  switch:  “Faster  is  better”  accurately  sums  up  the  pilots’  preferences  with  regard  to 
all  three  time-to-switch  functions  (switching  map  modes,  switching  chart  scales,  and  command 
lat/lon  repositions).  All  participants  rated  0.5  s  satisfactory  for  switching  chart  scales  and  for 
command  repositions,  and  although  0.5  s  was  not  an  option  for  the  map  mode  switches,  the  pilots 
would  undoubtedly  find  0.5  s  satisfactory  for  that  function  as  well.  All  tactical  pilots  rated  1  s  as 
satisfactory  for  switching  map  modes,  and  most  tactical  pilots  rated  1  s  as  satisfactory  for  scale 
switches  and  command  repositions.  All  but  one  nontactical  pilot  rated  1  s  as  satisfactory  for  all 
three  switch  functions.  Most  pilots  (tactical  and  nontactical)  rated  2  s  as  unsatisfactory  for  all  three 
functions.  All  but  one  pilot  rated  3  s  as  unsatisfactory  for  map  mode  switches,  and  although  3  s 
was  not  an  option  for  scale  switches  or  command  repositions,  these  pilots  would  certainly  find 
3  s  unsatisfactory  for  those  functions  as  well. 

Based  on  these  results,  the  authors  most  highly  recommend  a  0.5-s  time  to  switch  for  all  three 
functions.  If  0.5  is  cost-prohibitive,  however,  a  1-s  time  to  switch  would  be  acceptable.  This  study 
found  that  2  s  or  longer  is  unacceptable  for  any  of  the  demonstrated  functions.  Note  that  these 
results  reflect  what  pilots  want  to  see  on  a  fully  functional  cockpit  moving  map  display  (i.e.,  a 
moving  map  overlaid  with  all  necessary  mission  planning  symbols).  It  is  important  to  consider  that 
other  instruments  utilizing  the  cockpit  computer  will  contribute  to  latency,  so  it  may  be  necessary 
to  specify  a  faster  switch  time  (e.g.,  0.5  s)  to  ensure  satisfactory  performance  (e.g.,  1  s)  in  a 
fully  functional  cockpit  system. 

Data  Update  Rates:  It  is  interesting  that  the  “faster  is  better”  rule  does  not  apply  to  data  update 
rates.  Pilots  tended  to  prefer  the  demonstrated  15-Hz  displays  over  equivalent  20-Hz  displays, 
although  this  preference  was  apparently  related  to  aircraft  type  (see  Table  9).  Tactical  and  helicop¬ 
ter  pilots  showed  a  moderate  preference  for  the  15-Hz  display,  but  ASW  pilots  showed  absolutely 
no  preference  for  one  rate  or  the  other.  These  results  might  be  due  to  experiment  error  (e.g., 
inaccurately  simulating  data  update  rates  that  were  slower  than  the  computer’s  default  display  rate), 
they  might  have  been  due  to  pilot  conditioning  (expecting  faster  segments  of  the  demonstration 
first),  or  they  might  suggest  that  faster  data  update  rates  are  not  necessarily  better  for  cockpit 
moving  maps.  If  this  experiment  were  repeated,  investigators  should  present  randomized  data  update 
rates  to  ascertain  whether  the  lower  rate  is  really  more  acceptable. 

Based  on  the  results,  a  15-Hz  data  update  rate  seems  to  be  acceptable  for  the  TAMMAC 
system.  However,  moving  map  system  developers  must  ensure  that  this  rate  is  maintained  with  a 
full  load  on  the  computer  (i.e.,  when  all  other  aircraft  systems  that  utilize  the  map  computer  are 
running).  Also,  the  15-Hz  display  rate  must  be  maintained  whenever  all  necessary  overlays  have 
been  added  to  the  base  map  (e.g.,  threat  rings,  routes,  targets,  HAT,  etc.)  Pilots  in  this  study 
commented  that  this  is  not  the  case  with  existing  cockpit  map  systems — their  map  has  had  difficulty 
keeping  up  with  the  aircraft  position  after  all  the  necessary  symbols  were  overlaid,  which  was 
disorienting  and  unacceptable. 


4.2  Map  Positioning  Requirements 

4.2.1  Description 

Demo  #5:  Centered  vs.  Decentered  Modes  in  North-Up  and  Track-Up:  Demonstrated  four 
different  aircraft  cursor  positions  and  their  effect  on  the  moving  map  coverage:  centered  (aircraft 
cursor  in  the  center  of  the  display),  V4-up  (aircraft  cursor  25%  up  from  the  bottom  of  the  screen). 


# 
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‘/7-up  (14%  up  from  the  bottom),  and  bottom-of-screen.  Each  of  these  positions  was  demonstrated 
first  in  north-up  mode  (Fig.  6a-d)  then  track-up  (Fig.  7a-d).  For  each  variation,  the  computer 
played  back  a  recorded  sound  bite  that  identified  what  was  being  displayed:  “aircraft  is  centered;” 
“aircraft  is  one-quarter  up;”  etc. 

All  aircraft  programs  required  a  centered  cursor  option.  For  decentered  options,  preliminary 
TAMMAC  and  UH-IN  requirements  specified  V4-up  (listed  as  75%  in  App.  B,  item  31,  referencing 
the  aircraft  cursor  to  the  top  of  the  screen);  V-22  required  V7-up  (listed  as  86%  in  App.  B);  and  the 
decentered  position  in  current  F/A-18  and  AV-8B  display  systems  is  at  the  bottom  of  the  screen 
(listed  as  100%  in  App.  B). 


Fig.  6  —  Demo  5:  north-up  moving  map  (JOG  chart,  l;250k  scale),  (a)  aircraft  centered;  (b)  decentered  1/4-up; 
(c)  decentered  1/7-up;  and  (d)  decentered  bottom-of-screen 
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Fig.  7  — Demo  5:  track-up  moving  map  (JOG  chart,  l;250k  scale)  with  (a)  aircraft  centered;  (b)  decentered  1/4-up; 

(c)  decentered  1/7-up;  and  (d)  decentered  bottom-of-screen 


4.2.2  Results 

Table  10  presents  the  results  of  the  map  positioning  survey. 

North-Up:  As  could  be  expected,  all  pilots  preferred  a  centered  display  over  any  degree  of 
decenter  while  the  map  was  north-up.  Some  participants  found  a  slightly  decentered  position 
(74-up)  to  be  useful,  but  very  few  liked  Vv-up  in  a  north-up  mode,  and  no  pilots  liked  a  bottom- 
of-screen  position  for  north-up,  since  any  aircraft  headings  other  than  due-north  would  result  in  a 
significant  loss  of  forward-looking  map  information. 
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Table  10  —  Ratings  of  Map  Positions  (Demo  5);  1  =  Of  No  Use;  2  =  Not  Very  Useful; 
3  =  Of  Use;  4  =  Of  Considerable  Use;  5  =  Extremely  Useful 


DEMO 

DESCRIPTION 

AVERAGE  RATING 
(of  a  possible  5) 

DEMO 

DESCRIPTION 

AVERAGE  RATING 
(of  a  possible  5) 

NORTH-UP 

TRACK-UP 

NORTH-UP 

TRACK-UP 

Centered 

4.0 

4.5 

1/7  Up 

1.7 

3.7 

Tactical 

3.7 

4.5 

Tactical 

1.5 

3.6 

Helicopter 

4.3 

4.4 

Helicopter 

2.0 

4.1 

ASW 

4.2 

4.5 

ASW 

1.7 

3.2 

1/4  Up 

2.2 

Bottom 

■n 

2.8 

Tactical 

2.2 

Tactical 

2.7 

Helicopter 

2.9 

Helicopter 

2.8 

ASW 

1.8 

■EH 

ASW 

1.2 

3.2 

Track-Up:  The  majority  of  pilots  in  all  categories  found  both  centered  and  V4-up  positions  to 
be  extremely  useful  (most  ratings  were  4  or  5  on  a  5-point  scale)  in  track-up  mode.  Most  pilots  also 
found  the  Vv-up  position  to  be  of  some  use.  Fewer  pilots  liked  the  bottom-of-screen  position,  even 
in  track-up,  citing  a  loss  of  SA.  Many  pilots  commented  on  the  need  for  some  map  information 
behind  the  aircraft,  which  is  lost  in  a  bottom-of-screen  position.  Nearly  all  pilots  stated  a  preference 
for  track-up  over  north-up,  and  some  helicopter  pilots  stated  a  need  for  a  heading-up  mode.  The 
following  is  a  representative  sample  of  pilot  and  aircrew  comments  concerning  their  preferences 
with  regard  to  north-up  vs.  track-up  and  centered  vs.  decentered  map  positions: 

Tactical  pilots: 

•  “Prefer  track-up  for  most  tactical  applications.” 

•  “Don’t  like  bottom-of-screen  (position):  lose  SA  in  one  quadrant.” 

•  “Track-up,  V4-up  (from  the  bottom  of  the  screen)  gives  the  best  balance  of  information 
in  front  of  and  behind  the  aircraft.” 

•  “North-up  is  disorienting  in  flight,  but  a  good  tool  for  preflight  checks  or  waypoint 
insertion.” 

Helicopter  pilots: 

•  “Track-up  ...  allows  the  pilot  to  keep  essential  information  oriented  ahead  of  the  aircraft.” 

•  “For  helo,  track-up  isn’t  heading-up.  In  low-airspeed  flight,  it  would  be  better  to  show 
heading  up  to  minimize  ‘display  flopping’.” 

ASW  pilots: 

•  “Preferred  track-up,  'A  up  ...  If  you  could  keep  letters  upright,  this  would  be  perfect!” 

•  “Track-up  is  definitely  better.  There  is  utility  in  having  the  aircraft  at  V4  or  (at  the) 
bottom  if  you  need  a  bigger  picture  in  front  of  you.” 
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The  questionnaire  also  asked  pilots  if  they  would  want  to  have  control  over  the  degree  of 
decenter  for  their  aircraft  cursor  (e.g.,  to  change  it  from  V4-up  to  bottom-of-screen).  Of  the  30  pilots 
surveyed,  only  three  wanted  no  control  over  the  cursor  position.  Most  wanted  to  be  able  to  change 
the  aircraft  cursor  position  at  any  time  (during  mission  planning  or  in  the  cockpit),  as  shown  in 
Table  11.  Participants’  reasons  for  wanting  this  control  centered  on  their  need  for  improved  SA  and 
more  flexibility  to  address  the  wide  variety  of  pilot  preferences  and  tactical  situations. 

Two  helicopter  pilots  (UH-IN  and  AH-IW)  wanted  even  more  map  positioning  freedom  than 
the  demo  presented.  One  asked  for  “complete  freedom  of  position  using  a  joystick  or  (toggle) 
switch,”  and  the  other  called  for  decentered  cursors  to  the  “left  and  right  for  slewing  information 
(threat  rings,  etc.)  when  the  threat  is  known  to  be  parallel  to  the  course.”  One  of  the  three  pilots 
who  did  not  want  in-flight  control  over  map  positioning  suggested  customizing  the  default  position 
to  each  aircraft  platform  via  software. 


4.2.3  Summary  and  Recommendations 

Most  pilots  wanted  both  centered  and  decentered  cursor  positions  on  the  digital  moving  map 
display,  and  they  wanted  both  track-up  and  north-up  modes.  Helicopter  pilots  also  requested  a 
heading-up  mode  for  more  accurate  map  tracking  without  “display  flopping.”  Nearly  all  participants 
wanted  some  control  over  the  percentage  of  decenter  (e.g.,  75%  decentered  vs.  100%  decentered); 
most  wanted  control  both  in  mission  planning  and  in  the  cockpit. 

In  track-up  mode,  pilots  found  both  the  centered  position  and  the  ’A-up  position  to  be  extremely 
useful  and  the  Vr-up  position  was  rated  of  considerable  use.  The  bottom-of-screen  was  not  as 


Table  11  —  Pilot  Preferences  for  Control  of  Default 
Aircraft  Cursor  Position  Relative  to  Map 


OPTIONS  FOR  AIRCRAFT  CURSOR  POSITION 

#  PILOTS 

Preset  Default,  Unalterable 

3 

Preset  Default,  Pilot-Changeable: 

Change  in  Mission  Planning  Only 

0 

Change  in  Cockpit  Only 

4 

Change  in  Mission  Planning  or  Cockpit 

11 

No  Preference 

0 

Pilot-Selectable  Default: 

Set  in  Mission  Planning  Only 

0 

Set  in  Cockpit  Only 

2 

Set  in  Mission  Planning  or  Cockpit 

9 

No  Preference 

0 

No  Preference 

0 

No  Answer 

1 

TOTAL 

30 
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popular,  primarily  due  to  a  loss  of  SA  behind  the  aircraft,  but  it  was  still  rated  of  use.  A  few  pilots 
said  they  would  occasionally  switch  from  a  more  centered  position  to  a  bottom-of-screen  position, 
particularly  at  higher  speeds,  to  keep  as  much  map  in  front  of  the  aircraft  as  possible.  In  north-up 
mode,  all  pilots  favored  the  centered  position  (which  they  rated  of  considerable  use),  and  they 
found  all  decentered  positions  to  be  not  very  useful  or  of  no  use. 

Regardless  of  aircraft  cursor  position,  most  participating  pilots  stated  that  a  track-up  mode  is 
significantly  more  useful  to  them  than  north-up  for  improved  SA.  These  pilots’  opinions  support 
the  findings  of  Aretz  and  Wickens  (1992),  who  concluded  that  people  interpret  maps  more  easily 
in  a  track-up  alignment,  and  that  track-up  may  be  the  best  alternative  for  electronic  map  displays. 
Aretz  (1991)  explains  that  whenever  the  north-up  alignment  does  not  match  the  direction  of  travel 
(i.e.,  the  aircraft  is  flying  in  any  direction  other  than  due  north),  the  pilot  must  perform  a  mental 
rotation  to  accurately  interpret  the  map.  Earlier  studies  have  shown  that  the  time  to  accomplish  this 
necessary  mental  rotation  is  proportional  to  the  difference  in  alignment  between  the  two  reference 
systems — in  this  case,  the  difference  between  the  direction  of  flight  and  due  north  (e.g.,  Aretz  1988, 
1989;  Hintzman  et  al.  1981). 

Nevertheless,  Aretz  (1991)  found  that  pilots  may  perform  certain  tasks  (e.g.,  reconnaissance) 
more  effectively  with  a  north-up  map.  Harwood  (1989)  suggested  that  helicopter  pilots  should  use 
north-up  when  they  require  absolute  object  location  in  an  unfamiliar  environment.  These  studies 
support  our  recommendation  to  provide  both  modes  (north-up  and  track-up)  as  a  pilot-selectable 
option.  We  also  recommend  providing  pilot-selectable  aircraft  cursor  positions,  including  both 
centered  and  'A-up  positions,  at  a  minimum,  and  possibly  the  ’A-up  or  bottom-of-screen  position 
as  well. 


4.3  Zooming  Requirements 
4.3.1  Description 

All  four  demos  in  this  category  address  issues  pertaining  to  zooming  the  moving  map  display. 
The  first  demo  illustrated  the  difference  between  zooming  in  on  a  particular  chart  series  (e.g.,  JNC) 
vs.  switching  the  chart  to  a  new  series  (e.g.,  ONC).  The  second  demo  illustrated  three  variations  on 
a  2:1  zoom:  in  1  step,  in  8  steps,  and  as  a  continuum  (simulated  with  >30  steps).  The  third  demo 
illustrated  zoom-out  capabilities.  The  last  demo  in  the  zoom  category  illustrated  differences  in 
range  and  legibility  between  the  Compressed  Aeronautical  Chart  (CAC)  database  in  existing 
F/A-18  and  AV-8B  digital  map  systems  and  the  new,  joint-standard  CADRG  database  that  the 
TAMMAC  system  will  employ. 

In  each  demo,  the  computer  played  a  sound  bite  that  identified  what  was  being  displayed  (e.g., 
“JNC,  40  nmi  range;”  “zooming  in;”  “ONC,  20  nmi  range;”  “zooming  in;”  etc.).  In  the  last  demo, 
the  pilots  were  not  told  which  movie  loop  displayed  CAC  and  which  displayed  CADRG;  the  two 
databases  were  referred  to  as  “Chart  A”  and  “Chart  B.” 

Demo  #6:  Zoom  vs.  Chart  Series  Switches:  Demonstrated  all  aeronautical  chart  series  currently 
available  in  the  ADRG  database,  including  GNC,  JNC,  ONC,  TPC,  JOG,  and  TLM-50  (Fig.  8a-f). 
The  TLM-100  series  was  not  available  for  China  Lake.  For  each  series,  the  demo  simulated  a 
moving  map  display  (with  the  aircraft  centered  in  north-up  mode).  Pilots  first  evaluated  each 
individual  chart  series,  then  they  evaluated  the  capability  of  zooming  in  on  a  series  before  switching 
to  the  next  series. 
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■Demo  6:  centered,  north-up  displays:  (a)  GNC  before  (left,  160  nmi  range)  and  after  (right,  80  nmi  range) 
2:1  zoom  and  (b)  JNC  before  (left,  80  nmi  range)  and  after  (right,  40  nmi  range)  2:1  zoom 


Demo  #7:  Number  of  Steps  to  Zoom:  Pilots  evaluated  zooming  into  a  chart  by  2:1  in  a  single 
step  (Fig.  9),  in  8  steps  (Fig.  10),  and  as  a  continuous  zoom. 

Demo  #S:  Zoom-out  Capability:  Pilots  evaluated  zooming  out  2:1  over  chart  data  (Fig.  1 1)  and 
over  satellite  imagery  (Fig.  12).  Demonstrated  zoom-out  in  a  single  step  only. 

Demo  #9:  Range  Detail/Legibility  Issues:  Pilots  evaluated  the  display  ranges  (i.e.,  map 
coverage)  of  two  different  chart  series  (TPC  and  JOG)  for  the  current  chart  database  (CAC)  and  the 
new,  joint-standard  chart  database  (CADRG).  Figure  13a  depicts  the  two  TPC  displays.  Fig.  13b 
depicts  the  JOG  displays.  TPC  CAC  data  displays  20  nmi  of  chart  coverage  from  top  to  bottom  of 
the  screen  in  current  display  systems,  while  TPC  CADRG  data  would  display  15.2  nmi  on  the  same 
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Fig.  8  —  (cont.)  (c)  ONC  before  (left,  40  nmi  range)  and  after  (right,  20  nmi  range)  2:1  zoom  and  (d)  TPC  before 
(left,  20  nmi  range)  and  after  (right,  10  nmi  range)  2:1  zoom 


screen.  Similarly,  JOG  CAC  displays  10  nmi  on  the  current  system,  and  JOG  CADRG  would 
display  7.6  nmi. 

Pilots  also  evaluated  chart  feature  detail  and  legibility  for  CAC  and  CADRG,  again  for  both 
TPC  and  JOG  series.  CAC  data  is  designed  for  displays  with  a  resolution  of  128  pixels  per  inch, 
while  CADRG  is  designed  to  be  displayed  at  169  pixels  per  inch. 

4.3.2  Results 


Figure  14  summarizes  the  participants’  responses  to  each  individual  chart  series  in  Demo  #6. 
Each  bar  represents  the  average  rating  of  a  given  chart  series,  categorized  by  pilot  group  (tactical. 
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(cont.)  (e)  JOG  before  (left,  10  nmi  range)  and  after  (right,  5  nmi  range)  2:1  zoom  and  (f)  TLM-50 
before  (left,  2  nmi  range)  and  after  (right,  1  nmi  range)  2:1  zoom 


helicopter,  or  ASW,  as  listed  in  Table  5).  The  ratings  range  from  1  to  5,  where  1  =  of  no  use  and 
5  =  extremely  useful  (see  Table  1).  Table  3  provides  chart  scales,  display  ranges,  and  other  details 
for  each  chart  series. 

The  average  ratings  of  each  chart  series  (regardless  of  pilot  group)  ranged  between  3  (of  use) 
and  5  (extremely  useful).  In  order  of  usefulness:  TPC  (average  rating  4.7),  JOG  (4.6),  ONC  (4.0), 
TLM-50  (3.7),  JNC  (3.5),  and  GNC  (3.4). 

Most  participating  pilots  rated  “zooming  within  a  chart  series”  of  use  or  better.  Twelve  pilots 
(41%)  rated  it  extremely  useful,  and  the  average  rating  for  this  capability  for  each  pilot  group 
(tactical,  helicopter,  and  ASW)  was  4  (of  considerable  use). 


Fig.  9  —  Demo  7:  2:1  zoom  in  a  single  step,  TPC  charts,  centered,  north-up  displays,  before  (left,  20  nmi  range) 

and  after  (right,  10  nmi  range) 


The  pilots  were  given  three  choices  for  implementing  zooms:  (A)  “Only  zoom  up  to  scale  of 
next  chart  series,  then  switch  series  (e.g.,  zoom  a  TPC  by  no  more  than  2:1,  then  switch  to  JOG);” 
(B)  “Allow  zooms  beyond  scale  of  next  chart  series;”  or  (C)  no  preference.  Eighteen  pilots  (62%) 
preferred  option  A,  seven  pilots  (24%)  preferred  option  B,  and  four  pilots  (14%)  had  no  preference. 
Preferences  varied  somewhat  by  pilot  group,  as  shown  in  Fig.  15:  12  of  the  14  tactical  pilots  (86%) 
and  half  of  the  ASW  pilots  preferred  option  A;  only  one  of  each  of  these  groups  preferred  B.  On 
the  other  hand,  only  one-third  of  the  helicopter  pilots  preferred  option  A,  but  more  than  half  of  this 
group  preferred  B. 

Figure  16  summarizes  participants’  preferences  with  respect  to  number  of  steps  to  zoom  (Demo  #7). 
Most  pilots  preferred  the  continuous  zoom,  rating  it  of  considerable  use  (average  rating  3.9). 
Participants  commented  that  a  continuous  zoom  could  “tailor  the  zoom  factor  to  display  the  infor¬ 
mation  of  interest,”  and  that  it  “maintained  SA  in  a  controlled,  predictable,  and  fast  manner.”  One 
participant  said  the  continuous  change  allowed  him  to  “follow  the  zoom”  more  easily,  allowing  him 
to  keep  track  of  important  map  features  without  the  disorientation  that  sometimes  occurs  with  large 
zoom  increments.  Conversely,  a  few  pilots  said  the  continuous  zoom  was  too  slow  for  a  2:1  scale 
increase.  One  pilot  lost  his  “feel  for  the  range”  (i.e.,  the  amount  of  map  coverage  displayed,  in 
nautical  miles). 

The  eight-step  zoom  was  rated  nearly  as  useful  as  the  continuous  zoom  (average  rating  3.6). 
Participants  commented  that  it  provided  “more  control”  and  avoided  “inadvertently  zooming  in  too 
far.”  Several  said  the  eight-step  was  a  “nice  compromise  between  the  one-step  and  continuous 
zooms.”  However,  a  few  said  the  eight-step  was  labor  intensive  (e.g.,  pushing  a  button  eight  times 
to  get  a  2:1  zoom),  and  others  disliked  the  potential  of  “overshooting”  a  desired  zoom  factor  with 
too  many  button  pushes  if  they  were  in  a  hurry.  Several  suggested  that  a  four-step  zoom  would 
work  better  for  their  applications. 

The  one-step  zoom  was  much  less  desirable  than  the  other  two  zooms  (average  rating  2.7). 
Several  pilots  commented  that  it  was  “difficult  to  keep  track  of  where  everything  is  (on  the  map) 
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STEP  1  STEP  2  STEP  3  STEP  4 


STEP  5  STEP  6  STEP  7  STEP  8 


Fig,  10  —  Demo  7:  2:1  zoom  in  eight  steps,  TPC  charts,  centered,  north-up  displays 


and  where  you  zoomed  to.”  Many  participants  stated  they  would  simply  switch  charts  (e.g.,  from 
a  JNC  to  an  ONC)  rather  than  zoom  2:1  in  a  single  step,  and  others  concurred  that  a  single-step 
2:1  zoom  was  only  useful  if  the  next-larger  chart  series  was  not  available. 

Pilots  found  a  zoom-out  capability  to  be  of  considerable  use  over  both  of  the  demonstrated  base 
maps  (chart  and  imagery).  The  ratings  for  zoom-out  over  both  base  maps  were  almost  identical 
within  each  pilot  category,  so  Fig.  16  shows  the  average  ratings  of  zoom-out  over  chart  and 
imagery  for  each  category.  Participants  commented  that  a  zoom-out  capability  would  be  very  useful 
for  getting  a  “quick-look”  at  the  big  picture,  without  having  to  “process  a  different  set  of  reference 
symbols.”  One  pilot  mentioned  that  zooming  out  over  a  base  map  of  satellite  imagery  “would  be 
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Fig.  11  — Demo  8:  2:1  zoom-out  over  chart  (with  threat  ring  overlays),  TLM-50  charts,  centered,  north-up 
displays,  original  (left,  2  nmi  range)  and  zoomed  out  (right,  4  nmi  range) 
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Fig.  12  —  Demo  8:  2:1  zoom-out  over  imagery  (with  threat  ring  overlays),  SPOT  satellite  images,  centered, 
north-up  displays,  original  (left,  2  nmi  range)  and  zoomed  out  (right,  4  nmi  range) 


most  useful  and  informative  with  overlays  (e.g.,  threats,  navigational  symbology,  etc.).” 
Participants  listed  numerous  applications  that  would  be  supported  by  a  zoom-out,  including 
“big-picture  SA;”  target  acquisition;  flying  outbound  from  a  target;  high-altitude  transit  (climbout); 
post-mission  egress;  mission  aborts;  and  emergency  contingencies.  As  in  zooming  in,  many  pilots 
stated  they  would  want  to  be  able  to  return  to  the  pre-zoomed-out  map  display  quickly,  with  a 
single  button  push. 

Finally,  pilots  evaluated  the  differences  in  range  (coverage)  and  detail  (legibility)  for  the  current 
chart  database  (CAC)  and  the  new,  joint-standard  chart  database  (CADRG),  both  of  which  are 
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Fig.  13  — Demo  9:  evaluating  range  and  detail  of  current  (CAC)  and  future  (CADRG)  chart  databases  -  centered, 
north-up  displays  on  current  cockpit  map  displays:  (a)  TPC  charts  (left,  CAC:  20  nmi  range;  right,  CADRG:  15.2  nmi 
range)  and  (b)  JOG  charts  (left,  CAC:  10  nmi  range;  right,  CADRG:  7.6  nmi  range) 


compressed  forms  of  NIMA  ADRG.  The  major  differences  between  the  two  databases  are  resolution 
(CAC  is  stored  at  128  pixels  per  inch,  CADRG  is  stored  at  169  pixels  per  inch)  and  compression 
ratio  (CAC  =  48:1,  CADRG  =  55:1).  In  addition,  CADRG  has  been  filtered  somewhat  to  clarify  the 
image.  As  a  result,  it  was  expected  that  CADRG  would  produce  a  higher  quality  map  display  than 
CAC,  despite  its  greater  compression  ratio.  However,  the  concern  was  that  the  higher  resolution  of 
CADRG  would  noticeably  reduce  the  resultant  map  coverage  (assuming  the  display  screen  is  not 
upgraded).  For  example,  CAC  data  for  the  TPC  series  displays  20  nmi  of  chart  from  top  to  bottom 
of  the  screen  in  current  display  systems,  while  TPC  CADRG  data  would  display  15.2  nmi  on  the 
same  screen.  Similarly,  JOG  CAC  displays  10  nmi  on  the  current  system,  whereas  JOG  CADRG 
would  display  only  7.6  nmi. 
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Fig.  14  —  Chart  series  preferences 
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Fig.  15  —  Preferences  for  implementing  zooms:  (A)  only 
zoom  up  to  scale  of  next  chart  series,  then  switch  series  and 
(B)  allow  zooms  beyond  scale  of  next  chart  series 
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Fig.  16  —  Zoom  preferences 


These  differences  in  display  range  must  be  taken  into  consideration  when  designing  the  new 
cockpit  displays.  Therefore,  we  asked  pilots  to  evaluate  the  geographic  range  (map  coverage)  and 
chart  detail  (legibility)  for  CAC  and  CADRG  first  using  TPCs  then  JOGs.  As  shown  in  Fig.  17, 
70%  of  surveyed  pilots  preferred  CAC  over  CADRG  to  provide  optimum  range  for  both  TPC  and 
JOG  series.  Only  10%  of  pilots  preferred  CADRG  for  optimum  range  and  20%  had  no  preference. 
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Fig.  17  —  Preferences  for  (a)  chart  range  and  (b)  chart  legibility 


For  chart  detail  and  legibility,  33%  of  pilots  preferred  CAC,  24%  preferred  CADRG,  and  43%  had 
no  preference. 

Chart  type  A,  which  was  the  current  chart  database  (CAC),  was  clearly  preferred  over  type  B 
(CADRG)  with  respect  to  geographic  range  and  map  coverage.  However,  there  was  no  significant 
preference  for  one  chart  type  with  respect  to  detail  and  legibility. 


4.3.3  Summary  and  Recommendations 

Based  on  the  results  of  these  surveys,  implementing  the  following  zoom  options  is  recommended: 
a  continuous  zoom-in  (with  a  wheel  control),  a  four-step,  2:1  zoom-in  (as  opposed  to  eight-step  or 
one-step  zooms),  and  a  one-step  2:1  zoom-out.  The  four-step  zoom-in  should  allow  “buffered 
button  pushing”  so  the  pilot  could  hit  the  zoom-in  button  quickly  four  times  and  the  display  would 
“catch-up”  with  stepped  zooms.  This  seems  to  be  a  good  compromise  among  all  the  pilots’  stated 
requirements  for  controlled  zooms,  constant  SA,  rapid  implementation,  and  minimum  pilot  work¬ 
load.  Whether  using  the  continuous  or  stepped-zoom  option,  the  pilot  needs  to  be  able  to  return  to 
the  original  scale  with  a  single  button  push.  If  all  of  these  zoom  options  would  be  too  expensive 
to  implement  initially,  then  omitting  the  continuous  zoom  is  recommended  until  it  is  feasible. 

Zooms  in  either  direction  (in  or  out)  should  only  be  permitted  to  the  next  available  chart  series, 
at  which  point  the  computer  should  automatically  switch  series.  Therefore,  the  computer  will  need 
to  “know”  which  chart  series  have  been  loaded  in  the  cockpit  moving  map  computer. 

Finally,  the  altered  map  coverage  of  CADRG  (as  compared  to  CAC)  must  be  considered  in  the 
moving  map  display.  One  option  would  be  to  make  the  new  display  about  1.3  times  larger  (about 
5.3"  square,  compared  to  the  current  4")  to  accommodate  the  increased  resolution  of  CADRG.  If 
the  display  will  not  be  upgraded  in  TAMMAC,  then  consider  zooming  out  CADRG  by  about  0.8:1 
to  ensure  adequate  map  coverage  on  the  screen.  Pilots  are  accustomed  to  having  a  full  20  nmi  of 
coverage  when  they  choose  TPC  (for  example).  In  fact,  they  typically  refer  to  a  chart  series  in  terms 
of  its  coverage  (i.e.,  TPC  is  referred  to  as  a  “20  nmi  chart.”)  It  would  be  confusing  and  potentially 
dangerous  to  display  15.2  nmi  coverage  for  a  TPC  when  pilots  are  conditioned  to  viewing  20  nmi. 
Pilots  also  commented  (in  the  taped  interviews)  on  the  usefulness  of  having  an  integer  range  value 
(e.g.,  10  nmi,  20  nmi,  40  nmi,  etc.)  and  that  it  would  be  more  difficult  to  calculate  distances 
on-the-fly  with  non-integer  values  (e.g.,  7.6  nmi,  15.2  nmi,  30.4  nmi,  etc.). 
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4.4  Terrain  Elevation  Data 

4.4.1  Description 

Demo  #10:  Contour  Lines  V5.  Shaded  Contours:  Four  different  terrain  elevation  maps  were 
demonstrated:  contour  lines  and  gray-shaded  contours  in  2-D  (Fig.  18a, b)  and  in  3-D  (Fig.  18c, d). 
Contour  lines  were  drawn  at  32-m  contour  intervals,  and  gray-shaded  contour  maps  depicted  up  to 
16  gray  levels.  Participants  were  not  informed  of  these  values.  Aircraft  program  preferences  for 
contour  intervals  are  summarized  in  App.  B  (item  43),  but  there  are  no  stated  preferences  for  the 
number  of  gray-shading  levels. 


Fig.  18  —  Demo  10:  evaluating  two  pre.sentations  of  terrain  data,  contour  lines  vs.  shaded  contours.  All  include  height- 
above-terrain  overlays,  DIED  (yellow  =  aircraft  altitude  ±16  m;  red  =  all  altitudes  above  yellow):  2-D,  centered  display, 
track-up  ((a)  32  m;  (b)  16  gray  levels)  and  3-D,  perspective-view  display  ((c)  32  m;  (d)  16  gray  levels). 
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Demo  #11:  Contour  Intervals:  Four  different  contour  intervals  for  terrain  elevations  were 
demonstrated;  100  ft,  32  m,  64  m,  and  128  m  (Fig.  19a-d).  These  values  were  taken  from  the 
TAMMAC  Requirements  Database  (summarized  in  App.  B,  item  43),  in  which  the  V-22  program 
stated  a  requirement  for  contour  intervals  ranging  from  16  to  128  m,  and  TAMMAC  stated  a 
requirement  for  100  ft.  We  were  unable  to  process  16  m  contour  intervals  for  the  selected  area  of 
coverage  because  the  density  of  lines  was  too  great  for  the  software  (Arcinfo)  to  handle  at  the  range 
being  displayed  (10  nmi).  Participants  were  not  told  what  the  contour  intervals  were;  they  simply 
viewed  individual  movies  that  were  labeled  “Contour  Interval  A,”  "...  B,”  “...  C,”  and  “...  D.” 


Fig.  19  —  Demo  11:  evaluating  different  contour  intervals  for  terrain  data  (10  nmi  range),  DTED,  centered  display, 

track-up;  (a)  100  ft;  (b)  32  m;  (c)  64  m;  and  (d)  128  m 
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Demo  #12:  Sun-Angle  Shading  (2-D  and  3-D):  Several  sun-angle-shaded  terrain  maps  in  2-D 
and  3-D  were  demonstrated.  For  2-D  sun-angle  shading,  we  compared  displays  using  8  vs.  15  gray 
levels  (Fig.  20).  Participants  were  asked  to  select  which  half  of  one  continuous  movie  they  pre¬ 
ferred:  the  first  half,  which  used  eight  gray  levels,  or  the  second  half,  which  used  15  levels. 
Alternatively,  they  could  select  “no  preference.”  It  was  not  divulged  how  many  gray  levels  were 
represented  in  each  half. 

For  the  3-D  sun-angle-shaded  terrain,  displays  were  compared  using  15°  sun-angle  increments 
(i.e.,  13  steps  from  0°  to  180°:  0°,  15°,  30°,  45°,  ...  180°)  vs.  1°  increments  (i.e.,  181  steps:  0°, 
1°,  2°,  3°,  ...  180°).  Figures  21a-d  illustrate  shaded  terrain  for  four  sample  sun  angles:  30°,  75°, 
105°,  and  150°,  which  would  approximate  8  a.m.,  11  a.m.,  1  p.m.,  and  4  p.m.  (assuming  a  6  a.m. 
sunrise  and  6  p.m.  sunset).  Participants  were  asked  to  select  which  model  they  preferred:  “Movie 
A,”  which  used  15°  increments  or  “Movie  B,”  which  used  1°  increments. 

4.2.2  Results 

Figure  22  presents  aircrew  ratings  for  three  different  terrain  elevation  maps:  contour  lines, 
gray-shaded  contours,  and  sun-angle  shading,  in  both  2-D  (planimetric)  and  3-D  (perspective) 
views.  The  first  four  histogram  sets  in  Fig.  22  present  results  of  Demo  10  (contour  lines  vs.  gray- 
shaded  contours),  and  the  last  two  sets  present  results  of  Demo  12  (sun-angle-shaded  terrain). 

Terrain  map  types:  As  shown  in  Fig.  22,  participants  consistently  preferred  sun-angle-shaded 
terrain  maps  over  both  contour  lines  and  gray-shaded  contours.  Most  participants  ranked  gray- 
shaded  contours  as  their  second  choice.  Participants  rated  contour  lines  last,  with  the  exception  of 
helicopter  pilots,  who  preferred  contour  lines  over  shaded  contours  (in  the  2-D  view  only). 

Most  participants  did  not  show  an  overwhelming  preference  for  2-D  vs.  3-D  views  of  terrain. 
Tactical  pilots  showed  no  preference  at  all  for  2-D  vs.  3-D  contours  (lines  or  gray-shaded),  but  they 


Fig.  20  —  Demo  12:  evaluating  2-D  sun-angle-shaded  terrain  data  (10  nmi  range,  DTED,  centered  display, 
track-up)  using  8  gray  levels  (left)  and  15  gray  levels  (right) 
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Fig.  21  — Demo  12:  evaluating  3-D  sun-angle-shaded  terrain  data  (DTED,  perspective  view  display).  Times  shown  are 
based  on  6  a.m.  sunrise  (0°  sun  angle),  6  p.m.  sunset  (180°  sun  angle),  and  sun  directly  overhead  at  noon  (90° 
sun  angle):  (a)  8  a.m.,  30°  angle;  (b)  11  a.m.,  75°  angle;  (c)  1  p.m.,  105°  angle;  and  (d)  4  p.m.,  150°  angle. 


preferred  3-D  over  2-D  for  sun-angle  shading.  Helicopter  pilots  preferred  2-D  over  3-D  for  contour 
lines,  but  they  preferred  3-D  over  2-D  for  both  gray-shaded  maps  and  sun-angle  shading.  ASW 
pilots  preferred  3-D  over  2-D  for  both  contour  maps  and  gray-shaded  maps,  but  they  showed  no 
preference  at  all  for  2-D  vs.  3-D  sun-angle-shaded  maps. 

Overall,  participants  rated  both  2-D  and  3-D  contours  not  very  useful  (except  helicopter  pilots, 
who  rated  2-D  contours  of  use).  Participants  rated  2-D  and  3-D  gray-shaded  terrain  maps  of  use  and 
of  considerable  use,  respectively,  and  they  rated  both  2-D  and  3-D  sun- angle-shaded  terrain  maps  of 
considerable  use. 
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Fig.  22  —  Preferences  for  terrain  elevation  map  types  (Demos  #10  and  #12) 


Participants’  comments  clarify  why  the  majority  preferred  sun-angle-shaded  terrain  over  other 
models.  Tactical  and  ASW  participants  cited  enhanced  SA  and  anticipation  of  hazards: 

•  “Excellent  SA  builder  for  flying  in  terrain;  gives  a  feel  for  terrain  and  potential  hiding  spots.” 

•  “Increases  SA  and  ability  to  develop  an  evasive  plan  and  anticipate  likely  threat  locations.” 

•  2-D  useful  for  recognizing  distant  terrain  and  even  doing  radar  interpretation.  3-D  extremely 
useful  when  flying  low  altitudes  and  collating  (the)  display  to  outside  world.” 

•  “Allows  me  to  anticipate  rising  terrain  easily  in  times  of  poor  weather  or  at  night.” 

•  “Best  if  sun  angle  corresponds  to  actual  time  of  day.  Would  help  in  target  area  recognition.” 

Six  of  the  15  participating  tactical  pilots  stated  that  sun-angle-shaded  terrain  displays  would  be 
useful  for  pre-flight  and  mission  planning,  as  well  as  in-flight  navigation. 

Seven  of  the  nine  participating  helicopter  pilots  cited  the  sun-angle-shaded  display’s  realistic 
appearance: 

•  “Presents  terrain  information  to  the  pilot  clearly  ...  as  it  might  appear  to  him  out  of  the  cockpit.” 

•  “Day  or  night:  superb!  Allows  you  to  see  what  terrain  may  be  obscured  due  to  shadowing,  and 
if  you  can  find  a  hiding  place  . . .  This  capability  allows  you  to  more  effectively  plan  your  tactics, 
leaving  less  to  guesswork.  Being  able  to  see  what  an  area  will  look  like  is  very  useful.” 

•  “(For)  terrain  orientation,  determining  hidden  obstacles  ...  and  checkpoints  at  night  on  night 
vision  goggles.” 

Four  helicopter  pilots  suggested  that  moon-angle  shading  would  be  useful,  as  well  as  sun 
angles.  Processing  moon-angle  shading  would  be  equivalent  to  processing  sun  angles,  since  the 
computer  simply  operates  on  a  generic  light  source. 

At  least  one  participant  would  prefer  to  “freeze”  the  shading  at  some  optimum  sun  angle  to 
provide  the  most  detail,  rather  than  shade  the  terrain  according  to  the  actual  flight  time.  He  stated 
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in  a  taped  interview,  “as  you  change  the  sun  angle,  (the  display)  really  washes  out  (which  is)  the 
way  it  is  in  the  real  world.  But  if  I  knew  what  was  really  there,  that  may  give  me  more  insight  into 
what  I  can  do.”  When  asked  if  he  might  be  disoriented  by  seeing  one  sun  angle  out  of  the  cockpit 
window  and  another  on  the  display,  the  pilot  replied,  “I  don’t  know,  but  I’d  love  to  go  fly  it  and 
find  out!”  Such  comments  reinforce  a  need  to  rigorously  flight-test  these  capabilities  in  simulators 
or  trainers  and  define  exactly  how  they  will  be  utilized  so  the  final  implementation  will  best  meet 
user  demands. 

Participants  were  less  impressed  with  contour  lines  and  gray-shaded  contours;  six  of  15  tactical 
pilots  cited  interpretability  and  ease  of  use  as  major  criteria  for  rating  terrain  maps.  One  participant 
stated:  “Both  3-D  models  (contour  lines  and  gray-shading)  were  very  difficult  to  interpret  and  use;” 
another  stated:  “Gray  shading  was  much  easier  to  interpret  (than  contour  lines).”  Two  ASW  par¬ 
ticipants  described  the  contour  line  displays  as  “cluttered.”  Several  tactical  pilots  suggested  there 
might  be  more  potential  for  these  displays  if  they  were  rendered  with  greater  resolution  (this  might 
require  higher  resolution  terrain  data):  “The  gray  shading  could  have  been  useful  if  it  didn’t  look 
so  blocky;”  “Gray-shaded  3-D  terrain  needs  better  resolution;  it  could  be  useful  for  low  altitude 
navigation  and  terrain  avoidance.” 

Most  helicopter  pilots  preferred  contour  lines  over  gray-shaded  terrain,  probably  because,  as 
one  pilot  explained,  “contour  lines  are  more  familiar”  to  this  group.  Two  helicopter  pilots  suggested 
that  some  combination  of  shading  and  contours  would  be  preferable.  However,  several  helicopter 
pilots  mentioned  that  none  of  these  displays  conveyed  “depth”  or  the  “terrain’s  shape”  very  well, 
making  it  difficult  for  them  to  “pick  a  proper  valley  and  landing  zone.” 

The  following  paragraphs  address  more  specific  issues  for  three  of  the  terrain  displays:  preferred 
number  of  contour  intervals  (for  contour  line  displays),  number  of  gray  levels  (for  2-D  sun-angle 
displays),  and  number  of  sun-angle  increments  (for  3-D  sun-angle  displays). 

Contour  intervals:  As  shown  in  Fig.  23,  participants  rated  most  contour  intervals  not  very 
useful  or  of  use,  with  the  exception  of  helicopter  pilots,  who  rated  32-m  contours  of  considerable 
use.  On  average,  all  three  groups  (i.e.,  tactical,  helicopter,  and  ASW)  rated  64-m  contour  intervals 
of  use  and  128-m  intervals  slightly  lower.  Each  group  rated  32-m  contours  slightly  higher  than 
100-ft  contours,  even  though  the  contour  values  (32  m  and  100  ft)  are  very  close  and  produce 
almost  identical  contour  charts.  The  preference  is  probably  due  to  the  fact  that  the  32-m  display 
included  a  Height  Above  Terrain  (HAT)  model,  in  which  a  yellow  overlay  represented  all  terrain 
elevations  at  the  aircraft  altitude  ±16  m  and  a  red  overlay  represented  all  elevations  above  that.  The 


100  ft  32  m  64  m  128  m 


Fig.  23 — Preferences  for  number  of  contour  intervals  in  contour  line  displays  (Demo  #11) 
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100-ft  contour  chart  did  not  include  HAT.  A  later  section  of  this  report  discusses  HAT  overlays  in 
more  detail  (Demo  #13  in  the  Overlay  Data  section). 

Number  of  Gray  Levels  for  2-D  Sun-Angle  Shading:  As  shown  in  Table  12,  only  one  participant 
showed  any  preference  for  8  vs.  15  levels  of  gray  in  the  2-D  sun-angle-shaded  terrain  data.  How¬ 
ever,  this  lack  of  preference  could  change  depending  on  the  variability  of  the  data.  The  region  of 
China  Lake  that  was  displayed  is  relatively  flat,  with  only  minor  elevation  changes  around  the 
edges  (Fig.  20).  In  a  region  with  greater  terrain  variance,  the  number  of  gray  levels  required  for 
a  realistic  scene  may  be  more  significant.  For  example,  Foley  and  Van  Dam  (1984,  p.  594)  demonstrated 
that  64  gray  levels  were  required  to  adequately  represent  one  continuous-tone,  black-and-white 
photograph  (where  “adequate”  meant  that  the  resultant  image  appeared  to  be  continuous,  with  no 
visible  contouring  between  the  gray  shades). 

Other  images  might  require  more  than  64  gray  levels,  depending  on  the  variance  of  the  data  and 
the  desired  result.  Some  contour  effects  are  probably  acceptable  in  sun-angle-shaded  terrain  dis¬ 
plays,  and  for  most  of  these  displays,  eight  shades  of  gray  are  probably  sufficient.  Nevertheless, 
several  gray  levels  (e.g.,  8,  16,  32)  for  various  regions  (e.g.,  flat,  hilly,  very  mountainous)  should 
be  evaluated  before  making  any  final  recommendations  about  the  number  of  gray  levels  required 
for  satisfactory  sun-angle-shaded  terrain  in  cockpit  moving  map  displays. 

Sun-Angle  Increments  for  3-D  Sun-Angle  Shading:  As  shown  in  Fig.  24,  most  pilots  preferred 
the  sun-angle-shaded  terrain  model  with  1°  increments,  which  provided  a  continuous  transition  in 
shadows  and  lighting  from  sunrise  to  sunset.  Several  participants  preferred  the  more  discrete  model 
with  15°  increments,  and  several  others  indicated  no  preference.  A  few  pilots  indicated  that  1° 
increments  were  unnecessary  or  superfluous. 


4.4.3  Summary  and  Recommendations 

Based  on  these  results,  the  implementation  of  sun-angle-  (and  moon-angle-)  shaded  terrain 
maps  in  both  2-D  and  3-D  is  highly  recommended.  Pilots  should  have  a  choice  between  viewing 
a  preset,  “ideal”  sun  angle  (which  could  be  set  in  mission  planning)  or  viewing  the  terrain  display 
with  a  true  sun  angle  that  would  change  dynamically  in  flight  with  the  time  of  day.  Most  participants 
judged  contour  lines  and  gray-shaded  contour  charts  to  be  too  confusing,  too  cluttered,  and  too  hard 


Table  12  —  Preferences 
for  Number  of  Gray 
Levels  in  2-D  Sun- 
Angle-Shaded  Displays 
(Demo  #12) 
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Fig.  24  —  Preferences  for  sun-angle  increments  in  3-D 
sun-angle-shaded  displays  (Demo  #12) 
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to  interpret  for  most  applications.  (We  concede,  however,  that  our  contour  lines  and  gray-shaded 
contours  could  be  improved  with  superior  computer  graphics  software.) 

It  is  important  to  consider  the  vertical  accuracy  of  the  source  terrain  data  when  determining 
optimal  contour  intervals.  According  to  the  NIMA  publication  Digitizing  the  Future  (NIMA  1996), 
“the  information  content  of  DTED  is  approximately  equivalent  to  the  contour  information  repre¬ 
sented  on  a  1:250,000  scale  map.  Exploitation  at  larger  scales  must  consider  each  individual  cell’s 
accuracy  evaluation.”  In  addition,  NIMA’s  accuracy  objectives  for  DTED  are  as  follows:  “absolute 
vertical  accuracy  (i.e.,  the  uncertainty  in  elevation  of  a  point  with  respect  to  Mean  Sea  Level)  of 
±30  m,  and  absolute  horizontal  accuracy  (i.e.,  the  uncertainty  in  horizontal  position  of  a  point  with 
respect  to  the  current  World  Geodetic  System)  of  ±50  m.”  These  are  objectives;  the  DTED  Product 
Specification  (NIMA  1986)  gives  an  absolute  horizontal  accuracy  of  ±130  m  (the  DTED  Specifi¬ 
cation  cites  the  same  absolute  vertical  accuracy  of  ±30  m  as  Digitizing  the  Future).  These  values 
suggest  that  contour  lines  plotted  at  intervals  closer  than  30  m  would  give  a  false  impression  of 
vertical  accuracy  (i.e.,  elevation)  for  this  data  set,  and  contour  lines  closer  than  50-130  m  may  give 
a  false  impression  of  horizontal  accuracy  (i.e.,  geographic  position  on  the  Earth)  for  this  data  set. 

Sun-angle-shaded  terrain  displays  might  also  present  a  false  impression  of  accuracy,  and  pilots 
should  be  cautioned.  However,  it  is  suggested  that  there  is  an  intuitive  difference  between  viewing 
terrain  data  as  a  discrete  series  of  contours,  which  by  their  very  nature  imply  a  certain  degree  of 
accuracy,  vs.  a  rendered  surface  (such  as  the  sun-angle-shaded  model),  which  looks  more  realistic 
but  does  not  necessarily  imply  the  same  absolute  accuracy. 


4.5  Overlay  Data 

4.5.1  Description 

The  nexfthree  demos  examined  pilots’  responses  to  different  types  of  new,  mission-specific 
information  overlaid  on  traditional  charts  and  imagery.  Three  different  kinds  of  mission  information 
are  illustrated  in  this  set  of  demos:  Height-Above-Terrain  (HAT),  Clear  Line  Of  Sight  (CLOS),  and 
threat  intervisibility  (e.g.,  threat  rings). 

Demo  #13:  HAT  Over  Chart  or  Imagery:  Evaluated  usefulness  of  displaying  bi-color  HAT 
overlays  in  which  a  yellow  overlay  denoted  terrain  elevations  at  the  aircraft  altitude  ±16  m  and 
a  red  overlay  denoted  all  terrain  elevations  above  that.  Demo  #13  displayed  HAT  over  chart  data 
in  2-D  and  3-D  (Fig.  25a),  HAT  over  terrain  data  as  2-D  contour  lines  and  3-D  gridded  mesh 
(Fig.  25b),  and  HAT  over  satellite  imagery  in  3-D  only  (Fig.  25c).  The  same  model  could  be  used 
to  display  other  HAT  types,  including  Height-Above-Threshold,  -Target,  or  -Touchdown.  This 
coloration  was  intended  to  reduce  pilot  workload  in  interpreting  contours,  shaded  elevations,  and 
hypsographic  tinting. 

Demo  #14:  CLOS  Over  Imagery:  Evaluated  the  usefulness  of  a  CLOS  model  using  two  display 
windows  (Fig.  26).  The  larger  upper  window  simulated  a  moving  map  of  satellite  imagery  overlaid 
by  a  threat  ring,  with  the  aircraft  cursor  centered  over  a  north-up  display.  The  aircraft  and  threat 
symbols  were  highly  visible  over  the  satellite  imagery  in  this  view.  The  bottom  window  displayed 
a  profile  of  the  terrain  between  the  threat  and  the  aircraft  (in  this  case,  a  helicopter).  At  the  start 
of  the  demo,  the  helicopter  was  hidden  behind  a  mountain,  then  it  ascended  to  bring  the  threat  in 
sight.  As  soon  as  the  threat  was  in  sight  of  the  aircraft,  a  red  line  appeared  (in  both  views) 
connecting  the  aircraft  and  threat  symbols. 
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Fig.  25  —  Demo  13:  evaluating  HAT  in  2-D  (left-side  plots,  centered  display,  track-up)  and  3-D  views 
(right-side  plots,  perspective  view).  Yellow  overlay  =  elevations  at  aircraft  altitude  of  +  16  m;  red  overlay 
=  elevations  higher  than  aircraft  altitude:  (a)  chart  data  (JOG),  (b)  terrain  data  (DTED),  and  (c)  HAT 
overlaid  on  chart  data  and  satellite  imagery,  chart  data  (JOG,  left)  and  satellite  imagery  (SPOT,  right). 
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Fig.  26  —  Demo  14:  Clear  (or  Calculated)  Line-Of-Sight  (CLOS)  with  satellite  imagery  for  base  map,  target  out  of 

sight  (left)  and  in  sight  (right) 


Demo  #15:  Threat  Intervisihility:  Evaluated  four  different  ways  of  depicting  threats  over  chart 
and  imagery;  threat  rings  (Fig.  27a,  b),  pseudo-transparent  hatched  overlays  (Fig.  27c,  d),  threat 
rings  with  spokes  (Fig.  27e,  f),  and  a  3-D  threat  dome  (Fig.  27  g,  h).  For  each  representation,  the 
moving  map  depicted  the  aircraft  flying  into  the  threat  area,  at  which  point  the  white  aircraft  cursor 
and  the  yellow  threat  symbol  changed  color  to  red  to  indicate  danger. 


4.5.2  Results 

Figure  28  reflects  participants’  ratings  of  HAT  over  various  base  maps.  Pilots  rated  HAT  over 
imagery  highest  overall,  with  only  one  respon.se  below  a  score  of  3  {of  use).  This  may  be  due  in 
part  to  the  high  visual  contrast  between  the  black-and-white  imagery  and  the  red-and-yellow  HAT 
colors,  which  made  interpretation  particularly  easy. 

Responses  to  HAT  shading  over  aeronautical  charts  were  also  favorable.  Some  participants 
commented  that  the  HAT  symbology  would  be  useful  for  both  terrain  masking  and  terrain  avoid¬ 
ance.  Difficulty  in  interpretation  sometimes  arose  when  HAT  colors  blended  with  similar  chart 
colors  or  obscured  important  chart  information.  The  3-D  (perspective)  view  of  HAT  over  chart  data 
was  slightly  favored  over  the  comparable  2-D  (planimetric)  view,  despite  (or  perhaps  because  of) 
the  relative  novelty  of  3-D  chart  displays  to  many  participants. 

In  contrast,  displaying  HAT  over  a  contour-line  elevation  map  proved  to  be  the  least  effective 
way  to  provide  terrain  clearance  information  to  pilots.  The  lack  of  additional  cartography  to  provide 
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Fig.  27 — Demo  15:  threat  rings  (centered  display,  north-up,  2  nmi  range),  aircraft  outside  threat  range  (left-side 
plots)  and  inside  range  (right-side  plots):  (a)  TLM-50  chart  data  and  (b)  SPOT  satellite  imagery 


context,  as  well  as  the  general  difficulty  in  interpreting  contour  lines,  negated  some  of  the  benefits 
of  the  HAT  display  that  pilots  noted  in  other  views.  Participants  preferred  the  3-D  perspective  view 
(a  mesh  grid)  slightly  over  the  2-D  contour  lines  as  a  base  map  for  HAT. 

Overall,  many  pilots  noted  that  HAT  would  be  extremely  valuable  for  terrain  avoidance,  and 
a  few  commented  that  this  was  the  single  most  useful  feature  they  had  seen  in  all  of  our  map 
display  demonstrations.  HAT  enhanced  the  base  maps  by  boldly  highlighting  the  most  critical 
terrain  elevations — those  that  were  at  or  above  the  aircraft’s  current  altitude. 

Helicopter  and  tactical  pilots  rated  CLOS  of  considerable  use  (average  ratings  4.0  and  3.6, 
respectively),  while  Special  Operations  aircrew  rated  CLOS  barely  of  use  (2.7).  These  ratings 
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Fig.  27  —  (cont.)  pseudo-transparent  hatched  threat  areas  (centered  display,  north-up,  2  nmi  range),  aircraft  outside 
threat  range  (left-side  plots)  and  inside  range  (right-side  plots):  (c)  TLM-50  chart  data  and  (d)  SPOT  satellite  imagery 


probably  reflect  the  relative  importance  of  terrain  information  for  each  group’s  flight  needs.  Pilots’ 
comments  focused  on  the  novelty  of  this  display,  despite  similarities  to  an  instrument  approach 
plate.  CLOS  appeared  to  be  most  useful  when  specific  information  was  required  for  terrain  masking 
relative  to  a  single  target  or  threat. 


Figure  29  reflects  participants’  preferences  with  respect  to  threat  intervisibility  displays.  The 
results  are  split  into  two  categories,  according  to  the  type  of  base  map  on  which  the  threats  were 
presented.  In  the  first  category,  threat  symbols  were  drawn  over  black-and-white  satellite  imagery; 
in  the  second  category,  the  symbols  were  drawn  over  color  aeronautical  chart  data. 


For  both  categories,  participants  generally  preferred  the  simplest  representation — threat  rings — 
because  it  obscured  the  least  amount  of  the  base  map.  Pilots  reported  that  hatched  areas  and  circles 
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Fig.  27  —  (cont.)  threat  rings  with  spokes  (centered  display,  north-up,  2  nmi  range),  aircraft  outside  threat  range 
(left-side  plots)  and  inside  range  (right-side  plots);  (e)  TLM-50  chart  data  and  (f)  SPOT  satellite  imagery 


with  spokes  obscured  too  much  underlying  information  in  the  threat  area,  while  adding  little  additional 
information  or  warning  of  the  threat,  compared  to  open  rings.  We  would  have  liked  to  test  a  true 
transparent  threat  overlay,  which  would  have  tinted  the  threat  areas  without  obscuring  them,  but 
time  constraints  and  technical  limitations  prevented  such  a  test.  When  pilots  were  queried  as  to  how 
they  might  respond  to  such  a  display,  their  reactions  were  mixed,  and  most  participants  maintained 
their  preference  for  threat  rings.  However,  a  few  commented  that  a  simple  circular  ring  was  a  poor 
model  of  the  actual  geographic  distribution  of  the  threat,  and  that  aircraft  speed  and  surrounding 
terrain  should  be  taken  into  consideration. 

In  several  of  these  demonstrations,  the  threat  symbols  and  the  aircraft  cursor  changed  color 
from  yellow  to  red  when  the  simulated  aircraft  entered  the  threat  area.  Participants’  reactions  to  this 
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Fig.  27  —  (cont.)  3-D  threat  dome  over  satellite  imagery,  in  perspective  display:  (g)  aircraft  outside  threat  range,  threat 
in  sight  (left)  and  approaching  threat  (right)  and  (h)  aircraft  entering  the  threat  range,  just  outside  threat  range  (left) 
and  within  range  (right) 


feature  were  mixed.  Some  pilots  appreciated  the  additional  cue  that  they  were  within  the  threat 
range,  particularly  when  viewing  a  relatively  large-scale  map  with  only  an  edge  of  the  threat  ring 
visible.  Other  participants  preferred  simpler  representations,  with  as  few  colors  and  display  changes 
as  possible. 

In  addition  to  standard  2-D  views,  participants  viewed  one  3-D  representation  of  satellite 
imagery  with  a  threat  “dome”  appearing  above  the  terrain.  The  symbology  was  similar  to  the 
previous  2-D  displays.  Pilots’  reactions  were  mixed,  perhaps  because  of  the  novelty  of  the  display 
or  limitations  of  the  computer  graphics  that  were  used  to  depict  movement  into  the  threat  dome. 
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Fig.  28  —  Ratings  of  HAT  models  over  different  base  maps  (Demo  #13) 
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Fig.  29  —  Ratings  of  threat  symbology  over  satellite  imagery  and  scanned  chart  data 


Many  commented  that  this  3-D  view  was  difficult  to  interpret,  and  they  could  not  tell  when  they 
had  actually  entered  the  dome,  or  once  inside,  how  best  to  exit.  Several  commented  that  a  3-D 
threat  representation  would  be  very  helpful  in  mission  planning,  but  not  in  the  cockpit. 

The  use  of  satellite  imagery  as  a  base  map  was  novel  to  most  participants,  since  it  is  not  used 
in  current  Navy  aircraft  moving  map  displays,  yet  it  produced  a  generally  favorable  response.  In 
particular,  pilots  preferred  threat  rings  displayed  over  imagery  to  threats  displayed  over  charts.  This 
was  reflected  by  the  fact  that  most  participants  rated  “rings  over  imagery”  a  4  (of  considerable  use) 
or  5  (extremely  useful),  and  no  participant  rated  this  representation  less  than  3  (of  use).  By  comparison, 
fewer  participants  rated  “rings  over  chart”  a  4  or  5,  and  two  pilots  rated  this  a  2  (not  very  useful). 
This  is  probably  due  to  the  fact  that  colored  symbols  over  black-and-white  imagery  provide  greater 
contrast  than  the  same  symbols  over  multicolored  charts.  The  symbology  and  colors  of  the  threat 
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rings  often  confused  or  obscured  chart  symbology,  but  they  stood  out  well  against  the  monochrome 
satellite  imagery. 

4.5.3  Summary  and  Recommendations 

Many  pilots  in  the  survey  considered  HAT  to  be  a  very  useful  enhancement  to  standard  electronic 
charts,  but  not  all  participants  agreed.  Because  of  this  disparity,  it  is  recommended  that  HAT  be  a 
user-selectable  feature  that  can  be  turned  on  or  off,  depending  on  mission  requirements.  HAT 
enhances  satellite  imagery  very  effectively  by  adding  relative  altitude  information  to  the  display 
and  providing  a  “quick-look”  at  the  critical  terrain  surrounding  the  aircraft.  Overlaid  on  contour 
lines,  however,  HAT  does  little  to  aid  an  already  difficult-to-interpret  design. 

Average  ratings  by  both  tactical  and  helicopter  pilots  indicated  that  CLOS  would  be  of  considerable 
use  for  their  applications  (3.6  and  4.0,  respectively).  The  CLOS  model  appealed  most  to  helicopter 
pilots,  probably  due  to  its  utility  in  determining  terrain  masking  from  threats  and  targets.  Therefore, 
including  this  feature  for  helicopters  and  other  aircraft  that  would  benefit  from  advanced  terrain 
masking  capabilities  is  recommended. 

“Rings  only”  encapsulates  most  participants’  preferences  for  threat  symbology.  While  there 
were  considerable  rating  differences  among  the  other  three  threat  symbols,  depending  on  pilot 
categories  (i.e..  Tactical,  Helicopter,  Special  Operations),  nearly  all  participants  rated  rings  highly. 
Also,  many  pilots  preferred  imagery  to  charts  for  threat  overlays,  probably  because  of  fewer  visual 
conflicts  in  the  display.  The  3-D  representation  (threat  “dome”),  as  presented,  was  not  judged  to 
be  of  much  benefit  to  pilots.  A  more  sophisticated  rendering  of  the  3-D  dome  might  produce  more 
favorable  responses,  however. 


4.6  Vector  Moving  Map  Displays 

4.6.1  Description 

Demo  #i  6:  Vector  Moving  Map  Display:  This  demonstration  depicted  vector  charts  that  included 
many  of  the  cartographic  features  seen  in  previously  demonstrated  aeronautical  charts.  Vector  maps 
are  rendered  from  individually  stored  objects,  including  lines  (e.g.,  roads),  points  with  associated 
symbols  (e.g.,  airports),  text  features  (e.g.,  city  names),  and  areas  (e.g.,  shaded  metropolitan  areas 
or  tinted  areas  of  constant  elevation).  In  contrast,  the  charts  shown  in  previous  demonstrations  were 
scanned  in  their  entirety  from  paper  products,  so  individual  cartographic  features  were  not  individually 
accessible  or  manipulable. 

Participants  were  asked  to  evaluate  three  potential  benefits  of  a  vector  map  display:  (1)  the 
ability  to  keep  text  upright  as  the  aircraft  turned  (while  the  map  rotated  in  a  track-up  orientation, 
as  shown  in  Fig.  30);  (2)  the  ability  to  declutter  the  display  after  zooming  out  to  a  lower  resolution, 
effectively  decreasing  the  chart  scale  (Fig.  31a-c);  and  (3)  the  converse  of  declutter — adding  detail 
to  the  display  after  zooming  in  to  a  higher  resolution  (effectively  increasing  the  chart  scale). 


4.6.2  Results 

When  asked  to  assess  their  prior  experience  with  vector-type  map  displays,  25  out  of  30 
participants  said  they  had  limited  or  no  experience  with  this  type  of  map.  Nevertheless,  24  out  of 
30  participants  considered  the  demonstrated  vector  map  to  be  easily  interpretable,  and  nearly  all 
participants  rated  the  three  featured  capabilities  (keeping  text  upright,  selectively  decluttering,  and 
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Fig.  30  —  Demo  16:  evaluating  capability  of  vector  moving  map  to  di.splay  text  upright  in  track-up  mode  (centered 
display,  track-up,  20  nmi  range):  aircraft  heading  northeast  (left)  and  heading  southeast  (right) 


adding  detail)  very  highly,  as  shown  in  Fig.  32.  No  pilot  rated  any  of  these  features  less  than  3  (of 
use)  and  virtually  all  helicopter  pilots  gave  all  three  capabilities  the  highest  possible  rating  (extremely 
useful). 

When  asked  specifically  what  map  details  they  would  want  to  add  to  this  type  of  display, 
participants  gave  a  wide  variety  of  responses — from  navigational  information  and  terrain  data  to 
obstacle  indications.  No  single  type  of  information  was  predominant  in  their  requests. 


4.6.3  Summary  and  Recommendations 

Based  on  pilot  responses,  vector-type  maps  clearly  have  potential  for  improving  pilot  performance. 
Keeping  text  upright  in  a  track-up  orientation  and  selective  decluttering  are  clear  advantages  over 
current  systems.  Almost  all  pilots  wanted  to  have  a  choice  between  manually  decluttering  the  map 
and  using  an  automatic  declutter  mode  to  remove  extraneous  details.  Additionally,  if  vector 
maps  were  integrated  into  mission  planning,  the  result  may  be  a  superior  method  for  displaying 
spatial  information  and  improving  SA. 

Many  studies  have  linked  display  complexity  to  pilot  performance,  especially  in  terms  of  the 
pilot’s  ability  to  absorb  and  utilize  the  displayed  information  (e.g.,  Aretz  1988;  Schons  and  Wickens 
1993;  Wickens  and  Carswell  1995).  The  last  two  reports  found  that  visual  clutter  can  disrupt  the 
pilot’s  visual  attention,  resulting  in  greater  uncertainty  concerning  target  location.  Therefore,  a 
vector-based  map  display  with  declutter  capabilities  should  be  a  significant  improvement  over  the 
current,  relatively  static,  raster  map  displays. 

However,  there  are  two  potential  obstacles  to  effectively  implementing  vector-based  maps.  The 
first  is  pilot  training,  since  the  customized  quality  of  vector  maps  inevitably  make  them  look 
different  from  standard  aeronautical  charts.  In  effect,  pilots  must  acquire  new  cartographic  skills  to 
assist  them  in  configuring  their  maps  for  specific  mission  requirements.  The  second  obstacle 


Fig.  31 — Demo  16:  centered  display,  north-up:  (a)  increased  clutter  during  zoom-out,  20  nmi  range,  before  zoom-out 
(left)  and  40  nmi  range  after  2:1  zoom-out  (right);  (b)  declutter  capability  of  vector  maps  (e.g.,  removing  terrain 
elevation  contours),  after  initial  declutter  (left)  and  after  removing  rivers  and  some  text  (right) 


to  implementing  vector  maps  in  the  cockpit  is  technical,  since  many  cartographic  options  will 
have  to  be  handled  by  the  map  display  system  to  avoid  overburdening  the  pilot.  While  storage  and 
display  limitations  are  rapidly  being  overcome  by  advances  in  computer  technology,  the  problems 
associated  with  computer-aided  cartography  are  still  numerous.  Automated  cartography  is  a  com¬ 
plex  science  still  in  its  infancy,  and  automatically  generated  maps  often  lack  the  visual  quality  of 
maps  created  manually  by  trained  cartographers.  Further  basic  research  in  this  area  is  sorely  needed. 


Clearly,  vector  map  technology  should  be  pursued  for  advanced  mission  planning  and  cockpit 
displays.  Implementation  of  this  technology  should  be  carefully  tested  to  ensure  optimal  pilot 
performance  and  enhanced  mission  success. 
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Fig.  32  —  Ratings  of  demonstrated  vector  map  functions 


5.0  SUMMARY 

Map  designers  who  will  develop  advanced  maps  for  electronic  cockpit  displays  must  weigh  the 
benefits  of  cartographic  flexibility  against  pilot  workload.  Pilots  are  already  overwhelmed  by  an 
abundance  of  information  from  numerous  cockpit  displays,  electronic  or  otherwise.  A  cockpit  map 
system  must  be  capable  of  conveying  critical  information  concerning  navigation,  threats,  and  tar¬ 
gets  in  a  manner  that  is  easily  interpretable  under  often  stressful  conditions.  Our  recommendations 
concerning  specific  map  data  types  and  presentations  are  provided  throughout  this  paper,  under 
each  demonstration  category  heading. 

This  study  only  measured  pilot  and  aircrew  preferences,  not  actual  performance,  with  respect 
to  the  various  map  presentations.  Other  studies  have  shown  significant  discrepancies  between  sub¬ 
jective  preference  ratings  and  performance  measures;  often,  subjects  do  not  prefer  the  display  that 
produced  the  best  performance  (e.g.,  Merwin  and  Wickens  1993).  Therefore,  we  highly  recommend 
that  these  pilot  preference  results  be  used  in  conjunction  with  flight  performance  tests  in  realistic 
flight  simulators  to  ensure  optimal  pilot  performance  prior  to  the  development  and  implementation 
of  a  new  map  display  system. 
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Appendix  A 

SAMPLE  QUESTIONNAIRE 


Participant 
Survey  1: 
Survey  2: 
Survey  3: 
Survey  4: 
Survey  5: 
Survey  6: 
Survey  7: 
Survey  8: 
Survey  9: 
Survey  10: 
Survey  11: 
Survey  12: 
Survey  13: 
Survey  14: 
Survey  15: 
Survey  16: 


Information . 

Time  to  switch  map  modes . 

Time  to  switch  scales . 

Command  lat/lon  reposition . 

Display  update  rates . 

Decenter  positions . 

Zoom  vs.  chart  switches . 

Number  of  steps  to  zoom . 

Zoom-out  capability . 

Scale/range  issues . 

Terrain  elevation  contours . 

Terrain  elevation  contour  intervals 

Sun-angle  shading  (2-D,  3-D) . 

HAT  over  chart  or  imagery . 

CLOS  over  imagery  (2-D,  3-D)  .... 

Threat  intervisibility . . 

Vector  map  display . 
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59 
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INFORMATION  ABOUT  THE  PARTICIPANT: 


■v'-i 

r.-‘'-i 


£  Srati 


-•  v'<  L  >  - 


E-mail  address 


ji.'.'  *  z 


I.-  .-. 


'S  '-’C'  ■-’ 
fi'  .-t 


i 


O  Navy 

O  Marine  Corps 
O  Air  Force 
O  Army 


o 

A-10 

o 

AV-8B 

o 

F-14 

w 

F/A-18 

o 

P-3 

o 

S-3 

o 

UH-1N 

o 

V-22 

c 

Other: 

0  A-10 

0  AV-8B 

0  F-14 

C  F/A-18 

C  P-3 

C)  s-3 

C  UH-1N 

(  )  V-22 

(y  Other: 

t 


V*i' 


ight  fly^r 


-V'  ""j.. 


O  No  combat  experience 
O  Limited  combat  experience 
O  Experienced  combat  pilot 


Instructor 


Fj'.  '-V.-’VVj^.X  . 


Digital  moving-map  experience:' 


O  No  experience 
O  Familiar  with  concept 
O  Limited  experience  (e.g.,  simulator) 
O  Occasional  use  in  cockpit 

Current  use  and  very  experienced 


Interfti 


g|j 


mtiwajir- 
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TIME  TO  SWITCH  MAP  MODES 


Demonstrating  the  amount  of  time  to  switch  between  map  modes.  The  display 
switches  from  a  chart  to  satellite  imagery  to  terrain  elevation  to  a  data  frame 
{in  this  case,  a  reconnaisance  photograph).  Each  switch  is  timed  from  a 
simulated  pilot  request  until  the  display  changes. 

Note:  to  simulate  the  amount  of  time  to  perform  the  switch,  the  display  turns 
black  v/hen  the  request  is  made,  and  returns  to  Its  original  color  when  the 

change  is  implemented. 


Indicate  in  the  chart  below  which  switch  times  (A,  B,  and  /  or  C)  would  be 
satisfactory  for  your  applications,  or  if  NONE  of  them  would  be  satisfactory: 

O  Time  A 
O  Time  B 
O  Time  C 
O  None 


Rate  the  demonstrated  map  modes  (data  types)  on  a  scale  of  1  to  5  to 
reflect  how  useful  YOU  feel  each  would  be  for  YOUR  map  display  needs: 


1  =  Of  no  use 

2  =  Not  very  useful 

3  =  Of  use 

4  =  Of  considerable  use 

5  =  Extremely  useful 


1 

2 

3 

4 

5 

Chart  data: 

O 

o 

o 

o 

o 

Satellite  imagery: 

o 

o 

o 

o 

o 

Terrain  elevations: 

o 

o 

o 

o 

o 

Data  frames: 

o 

o 

o 

o 

o 
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TIME  TO  SWITCH  SCALES 


Demonstrating  the  amount  of  time  to  switch  between  chart  scales-  The  display 
switchesfrom  JOG  {-lOnmi  range)  to  TPC(~20nmO  and  back  to  JOG- 
switch  is  timed  from  a  simulated  pitot  request  until  the  display  changes. 

Note:  to  simulate  the  amount  of  time  to  perform  the  sv/itch,  the  display  turns 
black  when  the  request  is  made,  and  returns  to  its  original  color  v/hen  the 

change  is  implemented- 


Indicate  in  the  chart  below  which  switch  times  (A,  B,  and  /  or  C)  would  be 
satisfactory  for  your  applications,  or  if  NONE  of  them  would  be  satisfactory: 


O  Time  A 
O  Time  B 
O  Time  C 
O  None 
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Comparing  two  different  display  update  rates  for  varying  aircraft  speeds  and 
degrees  of  bank.  For  each  display  update  rate,  speeds  are  simulated  at  0, 90, 
200,  and  300  kts.  All  turn  rates  are  simulated  at  7deg  /  sec,  and  the  resulting 
degrees  of  bank  are  shown.  All  views  are  JOG  (-10  nmi  range  / 1 :250,000  scale) 
charts  in  track-up  mode. 


You  win  be  shown  two  different  display  update  rates  (“A"  and  "B”)  for  each 
airspeed.  After  viewing  both  update  rates  for  a  given  airspeed,  indicate  in  the 
chart  below  if  update  rate  A  and  /  or  B  was  satisfactory  for  your  applications,  or  if 
NEITHER  was  satisfactory; 


A 

B 

Neither 

0  kts: 

'o' 

O . 

. 6 . ^ 

90  kts  (30  deg  bank): 

o 

o 

o  ; 

200  kts  (52  deg  bank): 

o 

o 

o  ; 

300  kts  (62  deg  bank): 

o 

o 

O  I 

If  either  update  rate  was  NOT  satisfactory  for  one  or  more  of  the  demonstrated 
airspeeds,  please  explain  why... 
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This  demo  shows  4  positions  for  the  aircraft  cursor  in  north-up  and  track-up  modes. 


Rate  the  demonstrated  aircraft  cursor  positions  on  a  scale  of  1  to  5  to  reflect  how 

useful  YOU  feel  each  would  be  for  YOUR  map  display  needs; 

1 

2 

3 

4 

5 

North-up:  centered  in  screen:  O 

O 

o 

o 

o 

1/4  up  from  bottom:  O 

O 

o 

o 

o 

1/7  up  from  bottom:  O 

o 

o 

o 

O  1  =  Of  no  use 

bottom  of  screen:  O 

o 

o 

o 

Q  2-  Not  very  useful 

3  =  Of  use 

Track-up:  centered  in  screen:  O 

o 

o 

o 

O  4  =  Of  considerable  use 

1/4  up  from  bottom:  O 

o 

o 

o 

Q  5  =  Extremely  useful 

1/7  up  from  bottom:  Q 

o 

o 

o 

O 

bottom  of  screen:  O 

o 

o 

o 

o ;  .  .  .  . 

Comments  re;  your  preferred  options ... 

How  would  you  prefer  the  default 
decentered  position  to  be  set  for  your 
map  display? 


O  Preset  default  /  unalterable 
O  Preset  default  /  pilot-changable 
O  Pilot  selectable  default 
O  No  preference 


If  you  chose  plfot-changaole  or 
selectable,  where  should  this  be  set? 


O  In  mission  planning  only 
O  In  the  cockpit  only 
O  In  mission  planning  and  cockpit 
O  No  preference 
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This  tJemo  shows  the  difference  between  zooming  in  on  a  chart  and  switching  from 
one  chart  to  another.  It  also  shows  every  available  chart  series  (except  TLM-1 00, 
which  was  not  available  for  China  Lake). 


Rate  each  of  the  demonstrated  chart  series  on  a  scale  of  1  to  5  to  reflect  how 
useful  YOU  feel  each  would  be  for  YOUR  map  display  needs; 

^  ^  ^  ^  ^  1=Ofno  use 

GNC(1:5M):  O  O  O  O  O  2  =  Not  very  useful 

JNC(1:2M):  O  O  O  O  O  3  =  Of  use 

r\  4  =  Of  considerable  use 
'  5  =  Extreme  y  usefu 


GNC  (1:5M): 

o 

o 

o 

o 

o 

JNC  (1:2M): 

o 

o 

o 

o 

o 

ONC 

o 

o 

o 

o 

o 

TPC  (1:500k): 

o 

o 

o 

o 

o 

JOG  (1:250k): 

o 

o 

o 

o 

o 

TLM  (1:50k): 

o 

o 

o 

o 

o 

Rate  the  usefulness  of  zooming  within  a  chart  series: 

1  2  3  4  5 

Zooms  within  a  chart  series:  O  O  O  O  O 


In  what  situations  (if  any)  would  zooming  within  a  chart  series  be  useful  to  you? 


How  would  you  want  the  computer  to  zoom  within  a  chart  series? 

O  Only  zoom  up  to  scale  of  next  chart  series,  then  switch  series 
(e.g.,  zoom  a  TPC  by  no  more  than  2:1,  then  switch  to  JOG) 

O  Allow  zooms  beyond  scale  of  next  chart  series 
O  No  preference 


Which  (if  any)  of  the  zoomed  charts  became  too  blocky  or  difficult  to  read? 
O  GNC  O  JNC  O  ONC  O  TPC  O  JOG  O  TLM  O  none 
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Demonstrating  three  variations  on  a  2:1  zoom: 

(1)  zoom  in  a  single  step; 

(2)  zoom  in  8  steps; 

(3)  "continuous"  zoom. 

AH  views  are  north-up,  centered  mode. 


Rate  each  of  the  demonstrated  zoom  models  on  a  scale  of  1  to  5  to 
reflect  how  useful  YOU  feel  each  would  be  for  YOUR  map  display  needs; 

1  =  Of  no  use 

2  =  Not  very  useful 

4  =  Of  considerable  use 

5  =  Extremel  y  useful 


1 

2 

3 

4 

5 

Zoom  (2:1)  in  a  single  step: 

O 

o 

o 

o 

o 

Zoom  (2:1)  in  8  steps: 

O 

o 

o 

o 

o 

"Continuous"  zoom  (2:1): 

o 

o 

o 

o 

o 

How  would  you  like  to  see  these  zoom  controls  implemented  {e.g.,  hold  button 
down  until  reach  desired  zoom;  press  button -for  each  zoom  increment;  press 
another  button  to  return  to  original  scale;  etc  ...) 
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Demonstrating  the  difference  in  dispiay  range  be^veen  current  and  next- 
generation  chart  displays  (e.g,,  20  nmi  from  top  to  bottom  of  screen  for  current 
TPC  display  vs.  15.2  nmi  for  future  TPCj  and  1 0  nmi  for  current  JOG  display  vs. 
7.6  nmi  for  future  JOG),  There  is  a  trade-off  between  display  range  (nmi)  and 
chart  detail  /  legibility,  which  you  are  asked  to  evaluate. 


Which  TPC  coverage  provided  the  best  range  (coverage) 
for  your  applications? 


O  Chart  A 
O  Chart  B 
O  No  preference 


Which  TPC  coverage  provided  the  best  detail  /  legibility 
for  your  applications? 


O  Chart  A 
O  Chart  B 
O  No  preference 


Which  JOG  coverage  provided  the  best  range  (coverage) 
for  your  applications? 


O  Chart  A 
O  Chart  B 
O  No  preference 


Which  JOG  coverage  provided  the  best  detail  /  legibility 
for  your  applications? 


O  Chart  A 
O  Chart  B 
O  No  preference 


Would  you  prefer  a  default  (pre-zoom)  display  to  q  q  , ,  .  .. 

show  a  chart  with  greater  range  (but  less  detail)  ® 

or  ipore  detail  (but  less  range)?  O  More  detail  /  smaller  range 

O  No  preference 


60 


Lohrenz  et  al. 


TAMMAC  Digital  Map  Requirements  Study 


61 


Lohrenz  et  al. 


Demonstrating  realistic  sun-angle  shading  over  terrain  elevations  in  2-d  and  3-d. 

Shading  simuiates  the  shadows  over  the  ground  for  a  specific  time  of  day. 
Comparisons  are  made  between  different  shading  techniques  (irgray  levels)  and 
different  #  of  sun-angle  steps  for  a  whole  day  (sunrise  to  sunset). 


Rate  the  demonstrated  sun-angle  shaded  views  (2'd  and  3-d)  on  a  scale  of  1  to  5 
to  reflect  how  useful  YOU  feel  each  would  be  for  YOUR  map  display  needs: 

1  =  of  NO  use 

2  =  not  very  useful 

4  =  of  considerable  use 

5  =  extremely  useful 


2- d  sun  angles:  q 

3- d  sun  angles:  q 


Briefly  describe  how  sun  angle  shading  would  be  useful  to  you 


The  2-d  movie  demonstrated  two  different  #  gray  levels 
for  sun-angle  shading.  Which  model  would  you  prefer  for 
your  applications  (A  or  B)  ? 

The  3-d  movies  demonstrated  two  different  precisions  of » 
sun  angle  (i.e.,  the  #  steps  used  to  simulate  the  sun's 
movement  from  sunrise  to  sunset).  Which  would  you 
prefer  for  your  applications  (A  or  8)  ? 


O  2-d  model  A 
O  2-d  model  B 
O  No  preference 


O  3-d  movie  A 
O  3-d  movie  B 
O  No  preference 
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Demonstrating  Height  Above  Threshold  {HAT)  overlays  on  a  moving  map: 
Yellow  =  all  elevations  at  A/C  altitude  +/- 16  m;  Red  =  all  elevations  above  yellow. 
HAT  shown  on  chart,  terrain,  and  imagery;  in  planimetric  (2-d)  and  perspective  (3-d). 


yes  no 

Are  2  HAT  bands  /  colors  sufficient?  □  □ 

Are  these  HAT  colors  (yeilov/,  red)  OK?  □  □ 

If  you  answered  “no"  to  either  question,  please  comment  (e.g..  How  many  HAT 
bands  v/ouid  be  sufficient?  Which  colors  would  you  prefer?  Why?) 


Should  users  be  able  to  select  their  own  HAT  colors? 
O  No 

sssss  O  Yes  (in  mission  planning  only) 

O  Yes  (in  mission  planning  or  in  the  cockpit) 


Rate  the  demonstrated  HAT  models  on  a  scale  of  1  to  5,  to  reflect  how 
useful  YOU  feel  each  would  be  for  YOUR  map  display  needs; 

1  =  Of  no  use 

2  =  Not  very  useful 

3  =  Of  use 

4  =  Of  considerable  use 

5  =  Extremely  useful 


1 

2 

3 

4 

5 

HA  T  over  2-d  chart:  O 

O 

o 

o 

o 

HAT  over  3-d  chart:  O 

o 

o 

o 

o 

HA  T  over  3-d  terrain:  O 

o 

o 

o 

o 

HAT  over  2-d  terrain:  O 

o 

o 

o 

o 

HAT  over  3-d  imagery:  O 

o 

o 

o 

o 

HAT  over  2-d  imagery:  O 

o 

o 

o 

o 

How  might  you  use  this  information  (HAT  overlays) 


Demonstrating  Clear  Line  of  Sight  <CLOS)  over  planimetric  (2-d)  imagery 
in  one  display  window,  with  a  perspective  (S-d)  profile  view  of  CLOS 
displayed  in  a  second  window.  All  views  are  track-up,  centered  mode. 


Rate  the  demonstrated  CLOS  model  on  a  scale  of  1  to  S 
to  reflect  how  useful  YOU  feel  each  would  tse  for  YOUR 
map  display  needs: 

1  Of  no  use 

2  =  Not  very  useful 

3  =  Of  use 

4  s:  Of  considerable  use 

5  =  Extremely  useful 


2~d  imagery  with  3-d  CLOS  profile:  q 


Briefly  describe  how  you  might  use  the  information  presented  in  this  model 


TAMMAC  Digital  Map  Requirements  Study 
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Demonstrating  threat  intervisibility  in  2'd  (threat  rings,  spokes,  and  transparent 
color  overlays)  and  3-d  (domes).  Color  of  overlay  (rings,  spokes,  etc)  changes 
from  yellow  to  red  as  aircraft  enters  "danger  zone”.  Ail  views  are  north-up, 
centered  mode.  Sequence  is  demonstrated  for  satellite  imagery  and  chart  data. 


Rate  each  of  the  demonstrated  models  of  threat  intervjsibujty  on  a  scale  of  1  to  5 
to  reflect  bow  useful  YOU  feel  each  would  be  for  YOUR.map  display  needs; 

1  =  Of  no  use 

2  =  Not  very  useful 

3  ss  Of  use 

4  =  Of  considerable  use 

5  =  Extremely  useful 


Imagery 


Chart 


2-d  threat  rings:  Q  Q  Q  Q  Q 

2-d  threat  areas:  Q  O  O  O  O 

2- d  threat  spokes:  O  O  O  O  O 

3- d  threat  domes:  O  O  O  O  O 


2-d  threat  rings:  3  0  0 
2-d  threat  areas:  n 

2- d  threat  spokes:  Q  O  O 

3- d  threat  domes:  O  O  O 


o  o 


Can  you  suggest  any  better  representations  of  threat  intervisifaility?  if  so,  please 
describe  . 


Comments  re;  how  you  might  use  this  information  (threat  overlays)  in  2-d  view: 


Comments  re;  how  you  might  use  this  information  in  3-d  view; 
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VECTOR  MAP  DISPLAY 


***  NOTE:  THIS  SURVEY  iNCLUDES  2  PAGES  1 

Demonstrating  vector  map  capabilities  {combining  user-selected  layers  of  map 
data).  Demo  includes  2*d  and  3-d  views  and  illustrates  the  unique  capabilities  of 
vector  maps,  as  compared  to  the  current  raster  scanned  charts. 


Have  you  had  any  experience  v/ith  vector  map  displays,  similar  to  this? 

O  No  experience 
O  Familiar  with  concept 
O  Limited  experience 
O  Occasional  use 
O  Very  experienced 


How  easily  could  you  interpret  the  information  in  this  map  display? 
O  I  could  not  understand  it  at  all 
O  I  could  interpret  the  map  with  some  difficulty 
O  I  could  easily  interpret  the  map 


Rate  the  usefulness  of  the  following  vector  map  capabilities  on  a  scale  of  1  to  5, 
to  reflect  how  useful  YOU  feel  each  would  be  for  YOUR  map  display  needs; 

1  =  Of  no  use 

2  =  Not  very  useful 

3  =  Of  use 

4 -Of  considerable  use 
5  =  Extremely  useful 

1  2  3  4  5 

Display  upright  text  on  track-up  map:  O  O  O  O  O 

Declutter  (e.g.,  for  zoom-outs):  O  O  O  O  O 

Selectively  add  map  details:  O  O  O  O  O 

2- d  vector  map  with  HAT  overlays:  O  O  O  O  O 

3- d  vector  map  with  HAT  overlays:  O  O  O  O  O 


Hov/  do  you  think  decluttering  should  be  implemented? 
O  Automatically  (based  on  zoom  factor  and  preset  priorities 
O  Manually  (pilot-selectable) 

O  Choice  between  automatic  /  manual  declutter  modes 


Appendix  B 


HUMAN  FACTORS  MATRIX  OF 
DIGITAL  MAP  DATA  REQUIREMENTS 


APPENDIX  B.  MATRIX  OF  MOVING-MAP  ISSUES  ADDRESSED  BYNRL  STUDY 
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